pax_global_header00006660000000000000000000000064132371330210014505gustar00rootroot0000000000000052 comment=f0e082a974100ea6f0e4630994c340542e543cfe vowpal-wabbit-8.5.0.dfsg1/000077500000000000000000000000001323713302100153005ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/.editorconfig000066400000000000000000000020011323713302100177460ustar00rootroot00000000000000# See http://EditorConfig.org # Please, either use an editor supporting editorconfig # or set up your editor style according to this file manually # or use `astyle --lineend=linux --indent=spaces=2 *.cc *.h` # (see http://astyle.sourceforge.net/). # This is the top-most EditorConfig file root = true # Unix-style newlines with a newline ending every file [*] charset = utf-8 end_of_line = lf insert_final_newline = true trim_trailing_whitespace = true # C++ files [*.{cc,h}] indent_style = space indent_size = 2 indent_brace_style = Allman # Makefiles need tab indentation [Makefile] indent_style = tab [*.md] trim_trailing_whitespace = false # If not set this breaks class feature blocks [*.tt] insert_final_newline = false [*.cs] end_of_line = crlf indent_style = space indent_size = 4 insert_final_newline = false # The following line results in warning/error: # # { /// # # Does reverting to Allman [vw_clr/*.{cpp,h}] end_of_line = crlf indent_style = space indent_size = 4 indent_brace_style = Allman vowpal-wabbit-8.5.0.dfsg1/.gitignore000066400000000000000000000034711323713302100172750ustar00rootroot00000000000000Makefile.in cluster/Makefile.in cluster/Makefile acinclude.d/libtool.m4 acinclude.d/ltoptions.m4 acinclude.d/ltsugar.m4 acinclude.d/ltversion.m4 acinclude.d/lt~obsolete.m4 aclocal.m4 config.guess config.sub configure depcomp install-sh libtool ltmain.sh missing vowpalwabbit/Makefile.in vowpalwabbit/config.h.in config.h *.la *.lo vowpalwabbit/stamp-* Makefile config.status test/*.predict test/*.cmp *.log *.o *.a t_* bin vw .* vw.1 *.cache *.tgz test/*.tmp test/models test/topk.model .libs .deps autom4te.cache/ vowpalwabbit/active_interactor vowpalwabbit/vw cluster/spanning_tree *.d *.prev library/ezexample_predict library/ezexample_train library/library_example library/recommend library/gd_mf_weights library/test_search library/search_generate test/RunTests.last.times *.dSYM *~ *.pc *.pyc .DS_Store # build folders **/x64 **/obj **/Debug **/Release **/dll **/ipch **/target # VS files *.opensdf *.suo *.sdf *.vcxproj.user *.csproj.user *.so demo/mnist/infimnist demo/ocr/letter.* demo/ocr/run.* **/TestResults # IntelliJ *.idea *.iml # Ignore NuGet Packages *.nupkg # Ignore the packages folder **/packages # except build/, which is used as an MSBuild target. !**/packages/build/ # Uncomment if necessary however generally it will be regenerated when needed !**/packages/repositories.config # QT VW.config VW.creator VW.creator.user VW.files VW.includes # Python packaging python/.cache/** python/.eggs/** python/.tox/** python/build/** python/dist/** python/src/** python/vowpalwabbit.egg-info/** python/.coverage /COPYING /README /compile /cs/Serializer/Visitors/VowpalWabbitNativeVisitorExt.cs /cs/Serializer/VowpalWabbitDefaultMarshallerExt.cs /cs/unittest/RunTests.cs vowpalwabbit/vw.VC.VC.opendb vowpalwabbit/vw.VC.db /cs/cs/Serializer/VowpalWabbitDefaultMarshallerExt.cs /python/pylibvw.pyd /python/pylibvw.cp35-win_amd64.pyd vowpal-wabbit-8.5.0.dfsg1/.gitkeep000066400000000000000000000000001323713302100167170ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/.travis.yml000066400000000000000000000017411323713302100174140ustar00rootroot00000000000000language: cpp compiler: - gcc before_install: - sudo apt-get update -qq - sudo apt-get install -qq libboost-all-dev - sudo apt-get install maven - sudo pip install cpp-coveralls wheel # use miniconda for python package testing - wget https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh -O miniconda.sh; - bash miniconda.sh -b -p $HOME/miniconda - export PATH="$HOME/miniconda/bin:$PATH" - hash -r - conda config --set always_yes yes --set changeps1 no - conda update -q conda - conda create -q -n test-python27 python=2.7 nomkl numpy scipy scikit-learn script: - make all - make python - mvn clean test -f java/pom.xml - make test - make test_gcov --always-make - cd python - source activate test-python27 - pip install pytest readme_renderer - python setup.py check -mrs - python setup.py install - py.test tests - source deactivate - cd .. after_success: - coveralls --exclude lib --exclude tests --gcov-options '\-lp' vowpal-wabbit-8.5.0.dfsg1/AUTHORS000066400000000000000000000025131323713302100163510ustar00rootroot00000000000000John Langford ... and many others. The git log includes at least: Alekh Agarwal Ales Tamchyna Alex Grubb Alexander K. Hudek Alexander Trufanov Alexey Rodriguez Yakushev Anna Choromanska Ariel Faigon B. Scott Michel Ben Gimpert Ben Wing Bharath Krishnan Brian Muller Chris Quirk Dan Melamed Daniel Erenrich Daniel Hsu Doug Shore Eric Whyne Francesco Orabona Gabriel Synnaeve Gordon Rios Hal Daume III Harry Robertson HaveF Hui Gao Ivan Dyedov Jacob Hofman Jaimyoung Kwon Jake Hofman Jan Szumiec Jarrod R Parker John Langford Jon Morra Kai-Wei Chang Kristian Holsheimer Lihong Li Luis Pedro Coelho Lukas Zilka Luong Hoang Marko Asplund Marcel Laverdet Martin Popel Martin Thomas Matthew Hoffman Michael Brundage Michelangelo D'Agostino Monami Sharma Nick Nussbaum Nikos Karampatziakis Nirupama Chandrasekaran Olek Zendel Olivier Chapelle Parag Agrawal Paul Mineiro Paulius Klyvis Priska Herger Qiao Mu Rob Zinkov Roy Frostig Rukshan Batuwitage Sam Lendle Sam Steingold Selim Raboudi Sergei Matusevich Scott Graham Sharat Chikkerur Shravan M Narayanamurthy Siddhartha Sen Stephane Ross Stephen Merity Tim Cowlishaw Tom De Coninck Vaclav Petricek Vladimir Vladimirov Werner Fenchel (not really) Xavier Daull Yaroslav Halchenko Zach Mayer Zeno Gantner Zhen Qin vowpal-wabbit-8.5.0.dfsg1/COPYING000066400000000000000000000030601323713302100163320ustar00rootroot00000000000000Copyright © Microsoft Corp 2012-2014, Yahoo! Inc. 2007-2012, and many individual contributors. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of the Microsoft Corp nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. vowpal-wabbit-8.5.0.dfsg1/INSTALL000066400000000000000000000224511323713302100163350ustar00rootroot00000000000000When working off the master branch try just typing 'make'. For releases do the below. Installation Instructions ************************* Copyright (C) 1994, 1995, 1996, 1999, 2000, 2001, 2002, 2004, 2005, 2006 Free Software Foundation, Inc. This file is free documentation; the Free Software Foundation gives unlimited permission to copy, distribute and modify it. Basic Installation ================== Briefly, the shell commands `./configure; make; make install' should configure, build, and install this package. The following more-detailed instructions are generic; see the `README.md' file for instructions specific to this package. The `configure' shell script attempts to guess correct values for various system-dependent variables used during compilation. It uses those values to create a `Makefile' in each directory of the package. It may also create one or more `.h' files containing system-dependent definitions. Finally, it creates a shell script `config.status' that you can run in the future to recreate the current configuration, and a file `config.log' containing compiler output (useful mainly for debugging `configure'). It can also use an optional file (typically called `config.cache' and enabled with `--cache-file=config.cache' or simply `-C') that saves the results of its tests to speed up reconfiguring. Caching is disabled by default to prevent problems with accidental use of stale cache files. If you need to do unusual things to compile the package, please try to figure out how `configure' could check whether to do them, and mail diffs or instructions to the address given in the `README.md' so they can be considered for the next release. If you are using the cache, and at some point `config.cache' contains results you don't want to keep, you may remove or edit it. The file `configure.ac' (or `configure.in') is used to create `configure' by a program called `autoconf'. You need `configure.ac' if you want to change it or regenerate `configure' using a newer version of `autoconf'. The simplest way to compile this package is: 1. `cd' to the directory containing the package's source code and type `./configure' to configure the package for your system. Running `configure' might take a while. While running, it prints some messages telling which features it is checking for. 2. Type `make' to compile the package. 3. Optionally, type `make check' to run any self-tests that come with the package. 4. Type `make install' to install the programs and any data files and documentation. 5. You can remove the program binaries and object files from the source code directory by typing `make clean'. To also remove the files that `configure' created (so you can compile the package for a different kind of computer), type `make distclean'. There is also a `make maintainer-clean' target, but that is intended mainly for the package's developers. If you use it, you may have to get all sorts of other programs in order to regenerate files that came with the distribution. Compilers and Options ===================== Some systems require unusual options for compilation or linking that the `configure' script does not know about. Run `./configure --help' for details on some of the pertinent environment variables. You can give `configure' initial values for configuration parameters by setting variables in the command line or in the environment. Here is an example: ./configure CC=c99 CFLAGS=-g LIBS=-lposix *Note Defining Variables::, for more details. Compiling For Multiple Architectures ==================================== You can compile the package for more than one kind of computer at the same time, by placing the object files for each architecture in their own directory. To do this, you can use GNU `make'. `cd' to the directory where you want the object files and executables to go and run the `configure' script. `configure' automatically checks for the source code in the directory that `configure' is in and in `..'. With a non-GNU `make', it is safer to compile the package for one architecture at a time in the source code directory. After you have installed the package for one architecture, use `make distclean' before reconfiguring for another architecture. Installation Names ================== By default, `make install' installs the package's commands under `/usr/local/bin', include files under `/usr/local/include', etc. You can specify an installation prefix other than `/usr/local' by giving `configure' the option `--prefix=PREFIX'. You can specify separate installation prefixes for architecture-specific files and architecture-independent files. If you pass the option `--exec-prefix=PREFIX' to `configure', the package uses PREFIX as the prefix for installing programs and libraries. Documentation and other data files still use the regular prefix. In addition, if you use an unusual directory layout you can give options like `--bindir=DIR' to specify different values for particular kinds of files. Run `configure --help' for a list of the directories you can set and what kinds of files go in them. If the package supports it, you can cause programs to be installed with an extra prefix or suffix on their names by giving `configure' the option `--program-prefix=PREFIX' or `--program-suffix=SUFFIX'. Optional Features ================= Some packages pay attention to `--enable-FEATURE' options to `configure', where FEATURE indicates an optional part of the package. They may also pay attention to `--with-PACKAGE' options, where PACKAGE is something like `gnu-as' or `x' (for the X Window System). The `README.md' should mention any `--enable-' and `--with-' options that the package recognizes. For packages that use the X Window System, `configure' can usually find the X include and library files automatically, but if it doesn't, you can use the `configure' options `--x-includes=DIR' and `--x-libraries=DIR' to specify their locations. Specifying the System Type ========================== There may be some features `configure' cannot figure out automatically, but needs to determine by the type of machine the package will run on. Usually, assuming the package is built to be run on the _same_ architectures, `configure' can figure that out, but if it prints a message saying it cannot guess the machine type, give it the `--build=TYPE' option. TYPE can either be a short name for the system type, such as `sun4', or a canonical name which has the form: CPU-COMPANY-SYSTEM where SYSTEM can have one of these forms: OS KERNEL-OS See the file `config.sub' for the possible values of each field. If `config.sub' isn't included in this package, then this package doesn't need to know the machine type. If you are _building_ compiler tools for cross-compiling, you should use the option `--target=TYPE' to select the type of system they will produce code for. If you want to _use_ a cross compiler, that generates code for a platform different from the build platform, you should specify the "host" platform (i.e., that on which the generated programs will eventually be run) with `--host=TYPE'. Sharing Defaults ================ If you want to set default values for `configure' scripts to share, you can create a site shell script called `config.site' that gives default values for variables like `CC', `cache_file', and `prefix'. `configure' looks for `PREFIX/share/config.site' if it exists, then `PREFIX/etc/config.site' if it exists. Or, you can set the `CONFIG_SITE' environment variable to the location of the site script. A warning: not all `configure' scripts look for a site script. Defining Variables ================== Variables not defined in a site shell script can be set in the environment passed to `configure'. However, some packages may run configure again during the build, and the customized values of these variables may be lost. In order to avoid this problem, you should set them in the `configure' command line, using `VAR=value'. For example: ./configure CC=/usr/local2/bin/gcc causes the specified `gcc' to be used as the C compiler (unless it is overridden in the site shell script). Unfortunately, this technique does not work for `CONFIG_SHELL' due to an Autoconf bug. Until the bug is fixed you can use this workaround: CONFIG_SHELL=/bin/bash /bin/bash ./configure CONFIG_SHELL=/bin/bash `configure' Invocation ====================== `configure' recognizes the following options to control how it operates. `--help' `-h' Print a summary of the options to `configure', and exit. `--version' `-V' Print the version of Autoconf used to generate the `configure' script, and exit. `--cache-file=FILE' Enable the cache: use and save the results of the tests in FILE, traditionally `config.cache'. FILE defaults to `/dev/null' to disable caching. `--config-cache' `-C' Alias for `--cache-file=config.cache'. `--quiet' `--silent' `-q' Do not print messages saying which checks are being made. To suppress all normal output, redirect it to `/dev/null' (any error messages will still be shown). `--srcdir=DIR' Look for the package's source code in directory DIR. Usually `configure' can determine that directory automatically. `configure' also accepts some other, not widely useful, options. Run `configure --help' for more details. vowpal-wabbit-8.5.0.dfsg1/LICENSE000066400000000000000000000030601323713302100163040ustar00rootroot00000000000000Copyright © Microsoft Corp 2012-2014, Yahoo! Inc. 2007-2012, and many individual contributors. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of the Microsoft Corp nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. vowpal-wabbit-8.5.0.dfsg1/Makefile.am000066400000000000000000000050331323713302100173350ustar00rootroot00000000000000SUBDIRS = vowpalwabbit cluster library nobase_include_HEADERS = vowpalwabbit/allreduce.h \ vowpalwabbit/comp_io.h \ vowpalwabbit/config.h \ vowpalwabbit/example.h \ vowpalwabbit/action_score.h \ vowpalwabbit/feature_group.h \ vowpalwabbit/cb_explore.h \ vowpalwabbit/crossplat_compat.h \ vowpalwabbit/parse_example.h \ vowpalwabbit/floatbits.h \ vowpalwabbit/global_data.h \ vowpalwabbit/hash.h \ vowpalwabbit/io_buf.h \ vowpalwabbit/learner.h \ vowpalwabbit/loss_functions.h \ vowpalwabbit/parse_primitives.h \ vowpalwabbit/parser.h \ vowpalwabbit/simple_label.h \ vowpalwabbit/v_array.h \ vowpalwabbit/vw.h \ vowpalwabbit/vwdll.h \ vowpalwabbit/label_parser.h \ vowpalwabbit/multiclass.h \ vowpalwabbit/cost_sensitive.h \ vowpalwabbit/cb.h \ vowpalwabbit/v_hashmap.h \ vowpalwabbit/memory.h \ vowpalwabbit/vw_exception.h \ vowpalwabbit/vw_validate.h \ vowpalwabbit/multilabel.h \ vowpalwabbit/constant.h \ vowpalwabbit/ezexample.h noinst_HEADERS = vowpalwabbit/accumulate.h \ vowpalwabbit/autolink.h \ vowpalwabbit/baseline.h \ vowpalwabbit/bfgs.h \ vowpalwabbit/binary.h \ vowpalwabbit/bs.h \ vowpalwabbit/cache.h \ vowpalwabbit/cb.h \ vowpalwabbit/cb_algs.h \ vowpalwabbit/cb_explore.h \ vowpalwabbit/cbify.h \ vowpalwabbit/comp_io.h \ vowpalwabbit/constant.h \ vowpalwabbit/cost_sensitive.h \ vowpalwabbit/csoaa.h \ vowpalwabbit/ect.h \ vowpalwabbit/interactions.h \ vowpalwabbit/gen_cs_example.h \ vowpalwabbit/gd.h \ vowpalwabbit/gd_mf.h \ vowpalwabbit/interact.h \ vowpalwabbit/kernel_svm.h \ vowpalwabbit/lda_core.h \ vowpalwabbit/log_multi.h \ vowpalwabbit/lrq.h \ vowpalwabbit/mf.h \ vowpalwabbit/multiclass.h \ vowpalwabbit/network.h \ vowpalwabbit/nn.h \ vowpalwabbit/noop.h \ vowpalwabbit/oaa.h \ vowpalwabbit/boosting.h \ vowpalwabbit/parse_args.h \ vowpalwabbit/parse_example.h \ vowpalwabbit/parse_regressor.h \ vowpalwabbit/print.h \ vowpalwabbit/rand48.h \ vowpalwabbit/recall_tree.h \ vowpalwabbit/reductions.h \ vowpalwabbit/scorer.h \ vowpalwabbit/search.h \ vowpalwabbit/search_sequencetask.h \ vowpalwabbit/search_entityrelationtask.h \ vowpalwabbit/search_hooktask.h \ vowpalwabbit/search_multiclasstask.h \ vowpalwabbit/search_dep_parser.h \ vowpalwabbit/sender.h \ vowpalwabbit/stagewise_poly.h \ vowpalwabbit/topk.h \ vowpalwabbit/unique_sort.h \ vowpalwabbit/v_hashmap.h ACLOCAL_AMFLAGS = -I acinclude.d pkgconfigdir = $(libdir)/pkgconfig pkgconfig_DATA = libvw.pc libvw_c_wrapper.pc test: all cd test && ./RunTests -d -f -E 0.001 ../vowpalwabbit/vw ../vowpalwabbit/vw vowpal-wabbit-8.5.0.dfsg1/Makefile.in000066400000000000000000000765221323713302100173610ustar00rootroot00000000000000# Makefile.in generated by automake 1.15 from Makefile.am. # @configure_input@ # Copyright (C) 1994-2014 Free Software Foundation, Inc. # This Makefile.in is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY, to the extent permitted by law; without # even the implied warranty of MERCHANTABILITY or FITNESS FOR A # PARTICULAR PURPOSE. @SET_MAKE@ VPATH = @srcdir@ am__is_gnu_make = { \ if test -z '$(MAKELEVEL)'; then \ false; \ elif test -n '$(MAKE_HOST)'; then \ true; \ elif test -n '$(MAKE_VERSION)' && test -n '$(CURDIR)'; then \ true; \ else \ false; \ fi; \ } am__make_running_with_option = \ case $${target_option-} in \ ?) ;; \ *) echo "am__make_running_with_option: internal error: invalid" \ "target option '$${target_option-}' specified" >&2; \ exit 1;; \ esac; \ has_opt=no; \ sane_makeflags=$$MAKEFLAGS; \ if $(am__is_gnu_make); then \ sane_makeflags=$$MFLAGS; \ else \ case $$MAKEFLAGS in \ *\\[\ \ ]*) \ bs=\\; \ sane_makeflags=`printf '%s\n' "$$MAKEFLAGS" \ | sed "s/$$bs$$bs[$$bs $$bs ]*//g"`;; \ esac; \ fi; \ skip_next=no; \ strip_trailopt () \ { \ flg=`printf '%s\n' "$$flg" | sed "s/$$1.*$$//"`; \ }; \ for flg in $$sane_makeflags; do \ test $$skip_next = yes && { skip_next=no; continue; }; \ case $$flg in \ *=*|--*) continue;; \ -*I) strip_trailopt 'I'; skip_next=yes;; \ -*I?*) strip_trailopt 'I';; \ -*O) strip_trailopt 'O'; skip_next=yes;; \ -*O?*) strip_trailopt 'O';; \ -*l) strip_trailopt 'l'; skip_next=yes;; \ -*l?*) strip_trailopt 'l';; \ -[dEDm]) skip_next=yes;; \ -[JT]) skip_next=yes;; \ esac; \ case $$flg in \ *$$target_option*) has_opt=yes; break;; \ esac; \ done; \ test $$has_opt = yes am__make_dryrun = (target_option=n; $(am__make_running_with_option)) am__make_keepgoing = (target_option=k; $(am__make_running_with_option)) pkgdatadir = $(datadir)/@PACKAGE@ pkgincludedir = $(includedir)/@PACKAGE@ pkglibdir = $(libdir)/@PACKAGE@ pkglibexecdir = $(libexecdir)/@PACKAGE@ am__cd = CDPATH="$${ZSH_VERSION+.}$(PATH_SEPARATOR)" && cd install_sh_DATA = $(install_sh) -c -m 644 install_sh_PROGRAM = $(install_sh) -c install_sh_SCRIPT = $(install_sh) -c INSTALL_HEADER = $(INSTALL_DATA) transform = $(program_transform_name) NORMAL_INSTALL = : PRE_INSTALL = : POST_INSTALL = : NORMAL_UNINSTALL = : PRE_UNINSTALL = : POST_UNINSTALL = : build_triplet = @build@ host_triplet = @host@ subdir = . ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 am__aclocal_m4_deps = $(top_srcdir)/acinclude.d/ax_boost_base.m4 \ $(top_srcdir)/acinclude.d/ax_boost_program_options.m4 \ $(top_srcdir)/acinclude.d/ax_check_zlib.m4 \ $(top_srcdir)/acinclude.d/ax_cxx_compile_stdcxx_11.m4 \ $(top_srcdir)/acinclude.d/ax_pthread.m4 \ $(top_srcdir)/acinclude.d/libtool.m4 \ $(top_srcdir)/acinclude.d/ltoptions.m4 \ $(top_srcdir)/acinclude.d/ltsugar.m4 \ $(top_srcdir)/acinclude.d/ltversion.m4 \ $(top_srcdir)/acinclude.d/lt~obsolete.m4 \ $(top_srcdir)/configure.ac am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \ $(ACLOCAL_M4) DIST_COMMON = $(srcdir)/Makefile.am $(top_srcdir)/configure \ $(am__configure_deps) $(nobase_include_HEADERS) \ $(noinst_HEADERS) $(am__DIST_COMMON) am__CONFIG_DISTCLEAN_FILES = config.status config.cache config.log \ configure.lineno config.status.lineno mkinstalldirs = $(install_sh) -d CONFIG_HEADER = $(top_builddir)/vowpalwabbit/config.h CONFIG_CLEAN_FILES = libvw.pc libvw_c_wrapper.pc CONFIG_CLEAN_VPATH_FILES = AM_V_P = $(am__v_P_@AM_V@) am__v_P_ = $(am__v_P_@AM_DEFAULT_V@) am__v_P_0 = false am__v_P_1 = : AM_V_GEN = $(am__v_GEN_@AM_V@) am__v_GEN_ = $(am__v_GEN_@AM_DEFAULT_V@) am__v_GEN_0 = @echo " GEN " $@; am__v_GEN_1 = AM_V_at = $(am__v_at_@AM_V@) am__v_at_ = $(am__v_at_@AM_DEFAULT_V@) am__v_at_0 = @ am__v_at_1 = SOURCES = DIST_SOURCES = RECURSIVE_TARGETS = all-recursive check-recursive cscopelist-recursive \ ctags-recursive dvi-recursive html-recursive info-recursive \ install-data-recursive install-dvi-recursive \ install-exec-recursive install-html-recursive \ install-info-recursive install-pdf-recursive \ install-ps-recursive install-recursive installcheck-recursive \ installdirs-recursive pdf-recursive ps-recursive \ tags-recursive uninstall-recursive am__can_run_installinfo = \ case $$AM_UPDATE_INFO_DIR in \ n|no|NO) false;; \ *) (install-info --version) >/dev/null 2>&1;; \ esac am__vpath_adj_setup = srcdirstrip=`echo "$(srcdir)" | sed 's|.|.|g'`; am__vpath_adj = case $$p in \ $(srcdir)/*) f=`echo "$$p" | sed "s|^$$srcdirstrip/||"`;; \ *) f=$$p;; \ esac; am__strip_dir = f=`echo $$p | sed -e 's|^.*/||'`; am__install_max = 40 am__nobase_strip_setup = \ srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*|]/\\\\&/g'` am__nobase_strip = \ for p in $$list; do echo "$$p"; done | sed -e "s|$$srcdirstrip/||" am__nobase_list = $(am__nobase_strip_setup); \ for p in $$list; do echo "$$p $$p"; done | \ sed "s| $$srcdirstrip/| |;"' / .*\//!s/ .*/ ./; s,\( .*\)/[^/]*$$,\1,' | \ $(AWK) 'BEGIN { files["."] = "" } { files[$$2] = files[$$2] " " $$1; \ if (++n[$$2] == $(am__install_max)) \ { print $$2, files[$$2]; n[$$2] = 0; files[$$2] = "" } } \ END { for (dir in files) print dir, files[dir] }' am__base_list = \ sed '$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;s/\n/ /g' | \ sed '$$!N;$$!N;$$!N;$$!N;s/\n/ /g' am__uninstall_files_from_dir = { \ test -z "$$files" \ || { test ! -d "$$dir" && test ! -f "$$dir" && test ! -r "$$dir"; } \ || { echo " ( cd '$$dir' && rm -f" $$files ")"; \ $(am__cd) "$$dir" && rm -f $$files; }; \ } am__installdirs = "$(DESTDIR)$(pkgconfigdir)" \ "$(DESTDIR)$(includedir)" DATA = $(pkgconfig_DATA) HEADERS = $(nobase_include_HEADERS) $(noinst_HEADERS) RECURSIVE_CLEAN_TARGETS = mostlyclean-recursive clean-recursive \ distclean-recursive maintainer-clean-recursive am__recursive_targets = \ $(RECURSIVE_TARGETS) \ $(RECURSIVE_CLEAN_TARGETS) \ $(am__extra_recursive_targets) AM_RECURSIVE_TARGETS = $(am__recursive_targets:-recursive=) TAGS CTAGS \ cscope distdir dist dist-all distcheck am__tagged_files = $(HEADERS) $(SOURCES) $(TAGS_FILES) $(LISP) # Read a list of newline-separated strings from the standard input, # and print each of them once, without duplicates. Input order is # *not* preserved. am__uniquify_input = $(AWK) '\ BEGIN { nonempty = 0; } \ { items[$$0] = 1; nonempty = 1; } \ END { if (nonempty) { for (i in items) print i; }; } \ ' # Make sure the list of sources is unique. This is necessary because, # e.g., the same source file might be shared among _SOURCES variables # for different programs/libraries. am__define_uniq_tagged_files = \ list='$(am__tagged_files)'; \ unique=`for i in $$list; do \ if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ done | $(am__uniquify_input)` ETAGS = etags CTAGS = ctags CSCOPE = cscope DIST_SUBDIRS = $(SUBDIRS) am__DIST_COMMON = $(srcdir)/Makefile.in $(srcdir)/libvw.pc.in \ $(srcdir)/libvw_c_wrapper.pc.in AUTHORS INSTALL README compile \ config.guess config.sub install-sh ltmain.sh missing DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST) distdir = $(PACKAGE)-$(VERSION) top_distdir = $(distdir) am__remove_distdir = \ if test -d "$(distdir)"; then \ find "$(distdir)" -type d ! -perm -200 -exec chmod u+w {} ';' \ && rm -rf "$(distdir)" \ || { sleep 5 && rm -rf "$(distdir)"; }; \ else :; fi am__post_remove_distdir = $(am__remove_distdir) am__relativize = \ dir0=`pwd`; \ sed_first='s,^\([^/]*\)/.*$$,\1,'; \ sed_rest='s,^[^/]*/*,,'; \ sed_last='s,^.*/\([^/]*\)$$,\1,'; \ sed_butlast='s,/*[^/]*$$,,'; \ while test -n "$$dir1"; do \ first=`echo "$$dir1" | sed -e "$$sed_first"`; \ if test "$$first" != "."; then \ if test "$$first" = ".."; then \ dir2=`echo "$$dir0" | sed -e "$$sed_last"`/"$$dir2"; \ dir0=`echo "$$dir0" | sed -e "$$sed_butlast"`; \ else \ first2=`echo "$$dir2" | sed -e "$$sed_first"`; \ if test "$$first2" = "$$first"; then \ dir2=`echo "$$dir2" | sed -e "$$sed_rest"`; \ else \ dir2="../$$dir2"; \ fi; \ dir0="$$dir0"/"$$first"; \ fi; \ fi; \ dir1=`echo "$$dir1" | sed -e "$$sed_rest"`; \ done; \ reldir="$$dir2" DIST_ARCHIVES = $(distdir).tar.gz GZIP_ENV = --best DIST_TARGETS = dist-gzip distuninstallcheck_listfiles = find . -type f -print am__distuninstallcheck_listfiles = $(distuninstallcheck_listfiles) \ | sed 's|^\./|$(prefix)/|' | grep -v '$(infodir)/dir$$' distcleancheck_listfiles = find . -type f -print ACLOCAL = @ACLOCAL@ AMTAR = @AMTAR@ AM_DEFAULT_VERBOSITY = @AM_DEFAULT_VERBOSITY@ AR = @AR@ AUTOCONF = @AUTOCONF@ AUTOHEADER = @AUTOHEADER@ AUTOMAKE = @AUTOMAKE@ AWK = @AWK@ BOOST_CPPFLAGS = @BOOST_CPPFLAGS@ BOOST_LDFLAGS = @BOOST_LDFLAGS@ BOOST_PROGRAM_OPTIONS_LIB = @BOOST_PROGRAM_OPTIONS_LIB@ CC = @CC@ CCDEPMODE = @CCDEPMODE@ CFLAGS = @CFLAGS@ CPP = @CPP@ CPPFLAGS = @CPPFLAGS@ CXX = @CXX@ CXXCPP = @CXXCPP@ CXXDEPMODE = @CXXDEPMODE@ CXXFLAGS = @CXXFLAGS@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DLLTOOL = @DLLTOOL@ DSYMUTIL = @DSYMUTIL@ DUMPBIN = @DUMPBIN@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ ECHO_T = @ECHO_T@ EGREP = @EGREP@ EXEEXT = @EXEEXT@ FGREP = @FGREP@ GREP = @GREP@ HAVE_CXX11 = @HAVE_CXX11@ INSTALL = @INSTALL@ INSTALL_DATA = @INSTALL_DATA@ INSTALL_PROGRAM = @INSTALL_PROGRAM@ INSTALL_SCRIPT = @INSTALL_SCRIPT@ INSTALL_STRIP_PROGRAM = @INSTALL_STRIP_PROGRAM@ LD = @LD@ LDFLAGS = @LDFLAGS@ LIBOBJS = @LIBOBJS@ LIBS = @LIBS@ LIBTOOL = @LIBTOOL@ LIPO = @LIPO@ LN_S = @LN_S@ LTLIBOBJS = @LTLIBOBJS@ LT_SYS_LIBRARY_PATH = @LT_SYS_LIBRARY_PATH@ MAKEINFO = @MAKEINFO@ MANIFEST_TOOL = @MANIFEST_TOOL@ MKDIR_P = @MKDIR_P@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ OTOOL = @OTOOL@ OTOOL64 = @OTOOL64@ PACKAGE = @PACKAGE@ PACKAGE_BUGREPORT = @PACKAGE_BUGREPORT@ PACKAGE_NAME = @PACKAGE_NAME@ PACKAGE_STRING = @PACKAGE_STRING@ PACKAGE_TARNAME = @PACKAGE_TARNAME@ PACKAGE_URL = @PACKAGE_URL@ PACKAGE_VERSION = @PACKAGE_VERSION@ PATH_SEPARATOR = @PATH_SEPARATOR@ PTHREAD_CC = @PTHREAD_CC@ PTHREAD_CFLAGS = @PTHREAD_CFLAGS@ PTHREAD_LIBS = @PTHREAD_LIBS@ RANLIB = @RANLIB@ SED = @SED@ SET_MAKE = @SET_MAKE@ SHELL = @SHELL@ STRIP = @STRIP@ VERSION = @VERSION@ ZLIB_CPPFLAGS = @ZLIB_CPPFLAGS@ ZLIB_LDFLAGS = @ZLIB_LDFLAGS@ abs_builddir = @abs_builddir@ abs_srcdir = @abs_srcdir@ abs_top_builddir = @abs_top_builddir@ abs_top_srcdir = @abs_top_srcdir@ ac_ct_AR = @ac_ct_AR@ ac_ct_CC = @ac_ct_CC@ ac_ct_CXX = @ac_ct_CXX@ ac_ct_DUMPBIN = @ac_ct_DUMPBIN@ am__include = @am__include@ am__leading_dot = @am__leading_dot@ am__quote = @am__quote@ am__tar = @am__tar@ am__untar = @am__untar@ ax_pthread_config = @ax_pthread_config@ bindir = @bindir@ build = @build@ build_alias = @build_alias@ build_cpu = @build_cpu@ build_os = @build_os@ build_vendor = @build_vendor@ builddir = @builddir@ datadir = @datadir@ datarootdir = @datarootdir@ docdir = @docdir@ dvidir = @dvidir@ exec_prefix = @exec_prefix@ host = @host@ host_alias = @host_alias@ host_cpu = @host_cpu@ host_os = @host_os@ host_vendor = @host_vendor@ htmldir = @htmldir@ includedir = @includedir@ infodir = @infodir@ install_sh = @install_sh@ libdir = @libdir@ libexecdir = @libexecdir@ localedir = @localedir@ localstatedir = @localstatedir@ mandir = @mandir@ mkdir_p = @mkdir_p@ oldincludedir = @oldincludedir@ pdfdir = @pdfdir@ prefix = @prefix@ program_transform_name = @program_transform_name@ psdir = @psdir@ runstatedir = @runstatedir@ sbindir = @sbindir@ sharedstatedir = @sharedstatedir@ srcdir = @srcdir@ sysconfdir = @sysconfdir@ target_alias = @target_alias@ top_build_prefix = @top_build_prefix@ top_builddir = @top_builddir@ top_srcdir = @top_srcdir@ SUBDIRS = vowpalwabbit cluster library nobase_include_HEADERS = vowpalwabbit/allreduce.h \ vowpalwabbit/comp_io.h \ vowpalwabbit/config.h \ vowpalwabbit/example.h \ vowpalwabbit/action_score.h \ vowpalwabbit/feature_group.h \ vowpalwabbit/cb_explore.h \ vowpalwabbit/crossplat_compat.h \ vowpalwabbit/parse_example.h \ vowpalwabbit/floatbits.h \ vowpalwabbit/global_data.h \ vowpalwabbit/hash.h \ vowpalwabbit/io_buf.h \ vowpalwabbit/learner.h \ vowpalwabbit/loss_functions.h \ vowpalwabbit/parse_primitives.h \ vowpalwabbit/parser.h \ vowpalwabbit/simple_label.h \ vowpalwabbit/v_array.h \ vowpalwabbit/vw.h \ vowpalwabbit/vwdll.h \ vowpalwabbit/label_parser.h \ vowpalwabbit/multiclass.h \ vowpalwabbit/cost_sensitive.h \ vowpalwabbit/cb.h \ vowpalwabbit/v_hashmap.h \ vowpalwabbit/memory.h \ vowpalwabbit/vw_exception.h \ vowpalwabbit/vw_validate.h \ vowpalwabbit/multilabel.h \ vowpalwabbit/constant.h \ vowpalwabbit/ezexample.h noinst_HEADERS = vowpalwabbit/accumulate.h \ vowpalwabbit/autolink.h \ vowpalwabbit/baseline.h \ vowpalwabbit/bfgs.h \ vowpalwabbit/binary.h \ vowpalwabbit/bs.h \ vowpalwabbit/cache.h \ vowpalwabbit/cb.h \ vowpalwabbit/cb_algs.h \ vowpalwabbit/cb_explore.h \ vowpalwabbit/cbify.h \ vowpalwabbit/comp_io.h \ vowpalwabbit/constant.h \ vowpalwabbit/cost_sensitive.h \ vowpalwabbit/csoaa.h \ vowpalwabbit/ect.h \ vowpalwabbit/interactions.h \ vowpalwabbit/gen_cs_example.h \ vowpalwabbit/gd.h \ vowpalwabbit/gd_mf.h \ vowpalwabbit/interact.h \ vowpalwabbit/kernel_svm.h \ vowpalwabbit/lda_core.h \ vowpalwabbit/log_multi.h \ vowpalwabbit/lrq.h \ vowpalwabbit/mf.h \ vowpalwabbit/multiclass.h \ vowpalwabbit/network.h \ vowpalwabbit/nn.h \ vowpalwabbit/noop.h \ vowpalwabbit/oaa.h \ vowpalwabbit/boosting.h \ vowpalwabbit/parse_args.h \ vowpalwabbit/parse_example.h \ vowpalwabbit/parse_regressor.h \ vowpalwabbit/print.h \ vowpalwabbit/rand48.h \ vowpalwabbit/recall_tree.h \ vowpalwabbit/reductions.h \ vowpalwabbit/scorer.h \ vowpalwabbit/search.h \ vowpalwabbit/search_sequencetask.h \ vowpalwabbit/search_entityrelationtask.h \ vowpalwabbit/search_hooktask.h \ vowpalwabbit/search_multiclasstask.h \ vowpalwabbit/search_dep_parser.h \ vowpalwabbit/sender.h \ vowpalwabbit/stagewise_poly.h \ vowpalwabbit/topk.h \ vowpalwabbit/unique_sort.h \ vowpalwabbit/v_hashmap.h ACLOCAL_AMFLAGS = -I acinclude.d pkgconfigdir = $(libdir)/pkgconfig pkgconfig_DATA = libvw.pc libvw_c_wrapper.pc all: all-recursive .SUFFIXES: am--refresh: Makefile @: $(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps) @for dep in $?; do \ case '$(am__configure_deps)' in \ *$$dep*) \ echo ' cd $(srcdir) && $(AUTOMAKE) --foreign'; \ $(am__cd) $(srcdir) && $(AUTOMAKE) --foreign \ && exit 0; \ exit 1;; \ esac; \ done; \ echo ' cd $(top_srcdir) && $(AUTOMAKE) --foreign Makefile'; \ $(am__cd) $(top_srcdir) && \ $(AUTOMAKE) --foreign Makefile Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status @case '$?' in \ *config.status*) \ echo ' $(SHELL) ./config.status'; \ $(SHELL) ./config.status;; \ *) \ echo ' cd $(top_builddir) && $(SHELL) ./config.status $@ $(am__depfiles_maybe)'; \ cd $(top_builddir) && $(SHELL) ./config.status $@ $(am__depfiles_maybe);; \ esac; $(top_builddir)/config.status: $(top_srcdir)/configure $(CONFIG_STATUS_DEPENDENCIES) $(SHELL) ./config.status --recheck $(top_srcdir)/configure: $(am__configure_deps) $(am__cd) $(srcdir) && $(AUTOCONF) $(ACLOCAL_M4): $(am__aclocal_m4_deps) $(am__cd) $(srcdir) && $(ACLOCAL) $(ACLOCAL_AMFLAGS) $(am__aclocal_m4_deps): libvw.pc: $(top_builddir)/config.status $(srcdir)/libvw.pc.in cd $(top_builddir) && $(SHELL) ./config.status $@ libvw_c_wrapper.pc: $(top_builddir)/config.status $(srcdir)/libvw_c_wrapper.pc.in cd $(top_builddir) && $(SHELL) ./config.status $@ mostlyclean-libtool: -rm -f *.lo clean-libtool: -rm -rf .libs _libs distclean-libtool: -rm -f libtool config.lt install-pkgconfigDATA: $(pkgconfig_DATA) @$(NORMAL_INSTALL) @list='$(pkgconfig_DATA)'; test -n "$(pkgconfigdir)" || list=; \ if test -n "$$list"; then \ echo " $(MKDIR_P) '$(DESTDIR)$(pkgconfigdir)'"; \ $(MKDIR_P) "$(DESTDIR)$(pkgconfigdir)" || exit 1; \ fi; \ for p in $$list; do \ if test -f "$$p"; then d=; else d="$(srcdir)/"; fi; \ echo "$$d$$p"; \ done | $(am__base_list) | \ while read files; do \ echo " $(INSTALL_DATA) $$files '$(DESTDIR)$(pkgconfigdir)'"; \ $(INSTALL_DATA) $$files "$(DESTDIR)$(pkgconfigdir)" || exit $$?; \ done uninstall-pkgconfigDATA: @$(NORMAL_UNINSTALL) @list='$(pkgconfig_DATA)'; test -n "$(pkgconfigdir)" || list=; \ files=`for p in $$list; do echo $$p; done | sed -e 's|^.*/||'`; \ dir='$(DESTDIR)$(pkgconfigdir)'; $(am__uninstall_files_from_dir) install-nobase_includeHEADERS: $(nobase_include_HEADERS) @$(NORMAL_INSTALL) @list='$(nobase_include_HEADERS)'; test -n "$(includedir)" || list=; \ if test -n "$$list"; then \ echo " $(MKDIR_P) '$(DESTDIR)$(includedir)'"; \ $(MKDIR_P) "$(DESTDIR)$(includedir)" || exit 1; \ fi; \ $(am__nobase_list) | while read dir files; do \ xfiles=; for file in $$files; do \ if test -f "$$file"; then xfiles="$$xfiles $$file"; \ else xfiles="$$xfiles $(srcdir)/$$file"; fi; done; \ test -z "$$xfiles" || { \ test "x$$dir" = x. || { \ echo " $(MKDIR_P) '$(DESTDIR)$(includedir)/$$dir'"; \ $(MKDIR_P) "$(DESTDIR)$(includedir)/$$dir"; }; \ echo " $(INSTALL_HEADER) $$xfiles '$(DESTDIR)$(includedir)/$$dir'"; \ $(INSTALL_HEADER) $$xfiles "$(DESTDIR)$(includedir)/$$dir" || exit $$?; }; \ done uninstall-nobase_includeHEADERS: @$(NORMAL_UNINSTALL) @list='$(nobase_include_HEADERS)'; test -n "$(includedir)" || list=; \ $(am__nobase_strip_setup); files=`$(am__nobase_strip)`; \ dir='$(DESTDIR)$(includedir)'; $(am__uninstall_files_from_dir) # This directory's subdirectories are mostly independent; you can cd # into them and run 'make' without going through this Makefile. # To change the values of 'make' variables: instead of editing Makefiles, # (1) if the variable is set in 'config.status', edit 'config.status' # (which will cause the Makefiles to be regenerated when you run 'make'); # (2) otherwise, pass the desired values on the 'make' command line. $(am__recursive_targets): @fail=; \ if $(am__make_keepgoing); then \ failcom='fail=yes'; \ else \ failcom='exit 1'; \ fi; \ dot_seen=no; \ target=`echo $@ | sed s/-recursive//`; \ case "$@" in \ distclean-* | maintainer-clean-*) list='$(DIST_SUBDIRS)' ;; \ *) list='$(SUBDIRS)' ;; \ esac; \ for subdir in $$list; do \ echo "Making $$target in $$subdir"; \ if test "$$subdir" = "."; then \ dot_seen=yes; \ local_target="$$target-am"; \ else \ local_target="$$target"; \ fi; \ ($(am__cd) $$subdir && $(MAKE) $(AM_MAKEFLAGS) $$local_target) \ || eval $$failcom; \ done; \ if test "$$dot_seen" = "no"; then \ $(MAKE) $(AM_MAKEFLAGS) "$$target-am" || exit 1; \ fi; test -z "$$fail" ID: $(am__tagged_files) $(am__define_uniq_tagged_files); mkid -fID $$unique tags: tags-recursive TAGS: tags tags-am: $(TAGS_DEPENDENCIES) $(am__tagged_files) set x; \ here=`pwd`; \ if ($(ETAGS) --etags-include --version) >/dev/null 2>&1; then \ include_option=--etags-include; \ empty_fix=.; \ else \ include_option=--include; \ empty_fix=; \ fi; \ list='$(SUBDIRS)'; for subdir in $$list; do \ if test "$$subdir" = .; then :; else \ test ! -f $$subdir/TAGS || \ set "$$@" "$$include_option=$$here/$$subdir/TAGS"; \ fi; \ done; \ $(am__define_uniq_tagged_files); \ shift; \ if test -z "$(ETAGS_ARGS)$$*$$unique"; then :; else \ test -n "$$unique" || unique=$$empty_fix; \ if test $$# -gt 0; then \ $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ "$$@" $$unique; \ else \ $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ $$unique; \ fi; \ fi ctags: ctags-recursive CTAGS: ctags ctags-am: $(TAGS_DEPENDENCIES) $(am__tagged_files) $(am__define_uniq_tagged_files); \ test -z "$(CTAGS_ARGS)$$unique" \ || $(CTAGS) $(CTAGSFLAGS) $(AM_CTAGSFLAGS) $(CTAGS_ARGS) \ $$unique GTAGS: here=`$(am__cd) $(top_builddir) && pwd` \ && $(am__cd) $(top_srcdir) \ && gtags -i $(GTAGS_ARGS) "$$here" cscope: cscope.files test ! -s cscope.files \ || $(CSCOPE) -b -q $(AM_CSCOPEFLAGS) $(CSCOPEFLAGS) -i cscope.files $(CSCOPE_ARGS) clean-cscope: -rm -f cscope.files cscope.files: clean-cscope cscopelist cscopelist: cscopelist-recursive cscopelist-am: $(am__tagged_files) list='$(am__tagged_files)'; \ case "$(srcdir)" in \ [\\/]* | ?:[\\/]*) sdir="$(srcdir)" ;; \ *) sdir=$(subdir)/$(srcdir) ;; \ esac; \ for i in $$list; do \ if test -f "$$i"; then \ echo "$(subdir)/$$i"; \ else \ echo "$$sdir/$$i"; \ fi; \ done >> $(top_builddir)/cscope.files distclean-tags: -rm -f TAGS ID GTAGS GRTAGS GSYMS GPATH tags -rm -f cscope.out cscope.in.out cscope.po.out cscope.files distdir: $(DISTFILES) $(am__remove_distdir) test -d "$(distdir)" || mkdir "$(distdir)" @srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ topsrcdirstrip=`echo "$(top_srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ list='$(DISTFILES)'; \ dist_files=`for file in $$list; do echo $$file; done | \ sed -e "s|^$$srcdirstrip/||;t" \ -e "s|^$$topsrcdirstrip/|$(top_builddir)/|;t"`; \ case $$dist_files in \ */*) $(MKDIR_P) `echo "$$dist_files" | \ sed '/\//!d;s|^|$(distdir)/|;s,/[^/]*$$,,' | \ sort -u` ;; \ esac; \ for file in $$dist_files; do \ if test -f $$file || test -d $$file; then d=.; else d=$(srcdir); fi; \ if test -d $$d/$$file; then \ dir=`echo "/$$file" | sed -e 's,/[^/]*$$,,'`; \ if test -d "$(distdir)/$$file"; then \ find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ fi; \ if test -d $(srcdir)/$$file && test $$d != $(srcdir); then \ cp -fpR $(srcdir)/$$file "$(distdir)$$dir" || exit 1; \ find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ fi; \ cp -fpR $$d/$$file "$(distdir)$$dir" || exit 1; \ else \ test -f "$(distdir)/$$file" \ || cp -p $$d/$$file "$(distdir)/$$file" \ || exit 1; \ fi; \ done @list='$(DIST_SUBDIRS)'; for subdir in $$list; do \ if test "$$subdir" = .; then :; else \ $(am__make_dryrun) \ || test -d "$(distdir)/$$subdir" \ || $(MKDIR_P) "$(distdir)/$$subdir" \ || exit 1; \ dir1=$$subdir; dir2="$(distdir)/$$subdir"; \ $(am__relativize); \ new_distdir=$$reldir; \ dir1=$$subdir; dir2="$(top_distdir)"; \ $(am__relativize); \ new_top_distdir=$$reldir; \ echo " (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) top_distdir="$$new_top_distdir" distdir="$$new_distdir" \\"; \ echo " am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)"; \ ($(am__cd) $$subdir && \ $(MAKE) $(AM_MAKEFLAGS) \ top_distdir="$$new_top_distdir" \ distdir="$$new_distdir" \ am__remove_distdir=: \ am__skip_length_check=: \ am__skip_mode_fix=: \ distdir) \ || exit 1; \ fi; \ done -test -n "$(am__skip_mode_fix)" \ || find "$(distdir)" -type d ! -perm -755 \ -exec chmod u+rwx,go+rx {} \; -o \ ! -type d ! -perm -444 -links 1 -exec chmod a+r {} \; -o \ ! -type d ! -perm -400 -exec chmod a+r {} \; -o \ ! -type d ! -perm -444 -exec $(install_sh) -c -m a+r {} {} \; \ || chmod -R a+r "$(distdir)" dist-gzip: distdir tardir=$(distdir) && $(am__tar) | GZIP=$(GZIP_ENV) gzip -c >$(distdir).tar.gz $(am__post_remove_distdir) dist-bzip2: distdir tardir=$(distdir) && $(am__tar) | BZIP2=$${BZIP2--9} bzip2 -c >$(distdir).tar.bz2 $(am__post_remove_distdir) dist-lzip: distdir tardir=$(distdir) && $(am__tar) | lzip -c $${LZIP_OPT--9} >$(distdir).tar.lz $(am__post_remove_distdir) dist-xz: distdir tardir=$(distdir) && $(am__tar) | XZ_OPT=$${XZ_OPT--e} xz -c >$(distdir).tar.xz $(am__post_remove_distdir) dist-tarZ: distdir @echo WARNING: "Support for distribution archives compressed with" \ "legacy program 'compress' is deprecated." >&2 @echo WARNING: "It will be removed altogether in Automake 2.0" >&2 tardir=$(distdir) && $(am__tar) | compress -c >$(distdir).tar.Z $(am__post_remove_distdir) dist-shar: distdir @echo WARNING: "Support for shar distribution archives is" \ "deprecated." >&2 @echo WARNING: "It will be removed altogether in Automake 2.0" >&2 shar $(distdir) | GZIP=$(GZIP_ENV) gzip -c >$(distdir).shar.gz $(am__post_remove_distdir) dist-zip: distdir -rm -f $(distdir).zip zip -rq $(distdir).zip $(distdir) $(am__post_remove_distdir) dist dist-all: $(MAKE) $(AM_MAKEFLAGS) $(DIST_TARGETS) am__post_remove_distdir='@:' $(am__post_remove_distdir) # This target untars the dist file and tries a VPATH configuration. Then # it guarantees that the distribution is self-contained by making another # tarfile. distcheck: dist case '$(DIST_ARCHIVES)' in \ *.tar.gz*) \ GZIP=$(GZIP_ENV) gzip -dc $(distdir).tar.gz | $(am__untar) ;;\ *.tar.bz2*) \ bzip2 -dc $(distdir).tar.bz2 | $(am__untar) ;;\ *.tar.lz*) \ lzip -dc $(distdir).tar.lz | $(am__untar) ;;\ *.tar.xz*) \ xz -dc $(distdir).tar.xz | $(am__untar) ;;\ *.tar.Z*) \ uncompress -c $(distdir).tar.Z | $(am__untar) ;;\ *.shar.gz*) \ GZIP=$(GZIP_ENV) gzip -dc $(distdir).shar.gz | unshar ;;\ *.zip*) \ unzip $(distdir).zip ;;\ esac chmod -R a-w $(distdir) chmod u+w $(distdir) mkdir $(distdir)/_build $(distdir)/_build/sub $(distdir)/_inst chmod a-w $(distdir) test -d $(distdir)/_build || exit 0; \ dc_install_base=`$(am__cd) $(distdir)/_inst && pwd | sed -e 's,^[^:\\/]:[\\/],/,'` \ && dc_destdir="$${TMPDIR-/tmp}/am-dc-$$$$/" \ && am__cwd=`pwd` \ && $(am__cd) $(distdir)/_build/sub \ && ../../configure \ $(AM_DISTCHECK_CONFIGURE_FLAGS) \ $(DISTCHECK_CONFIGURE_FLAGS) \ --srcdir=../.. --prefix="$$dc_install_base" \ && $(MAKE) $(AM_MAKEFLAGS) \ && $(MAKE) $(AM_MAKEFLAGS) dvi \ && $(MAKE) $(AM_MAKEFLAGS) check \ && $(MAKE) $(AM_MAKEFLAGS) install \ && $(MAKE) $(AM_MAKEFLAGS) installcheck \ && $(MAKE) $(AM_MAKEFLAGS) uninstall \ && $(MAKE) $(AM_MAKEFLAGS) distuninstallcheck_dir="$$dc_install_base" \ distuninstallcheck \ && chmod -R a-w "$$dc_install_base" \ && ({ \ (cd ../.. && umask 077 && mkdir "$$dc_destdir") \ && $(MAKE) $(AM_MAKEFLAGS) DESTDIR="$$dc_destdir" install \ && $(MAKE) $(AM_MAKEFLAGS) DESTDIR="$$dc_destdir" uninstall \ && $(MAKE) $(AM_MAKEFLAGS) DESTDIR="$$dc_destdir" \ distuninstallcheck_dir="$$dc_destdir" distuninstallcheck; \ } || { rm -rf "$$dc_destdir"; exit 1; }) \ && rm -rf "$$dc_destdir" \ && $(MAKE) $(AM_MAKEFLAGS) dist \ && rm -rf $(DIST_ARCHIVES) \ && $(MAKE) $(AM_MAKEFLAGS) distcleancheck \ && cd "$$am__cwd" \ || exit 1 $(am__post_remove_distdir) @(echo "$(distdir) archives ready for distribution: "; \ list='$(DIST_ARCHIVES)'; for i in $$list; do echo $$i; done) | \ sed -e 1h -e 1s/./=/g -e 1p -e 1x -e '$$p' -e '$$x' distuninstallcheck: @test -n '$(distuninstallcheck_dir)' || { \ echo 'ERROR: trying to run $@ with an empty' \ '$$(distuninstallcheck_dir)' >&2; \ exit 1; \ }; \ $(am__cd) '$(distuninstallcheck_dir)' || { \ echo 'ERROR: cannot chdir into $(distuninstallcheck_dir)' >&2; \ exit 1; \ }; \ test `$(am__distuninstallcheck_listfiles) | wc -l` -eq 0 \ || { echo "ERROR: files left after uninstall:" ; \ if test -n "$(DESTDIR)"; then \ echo " (check DESTDIR support)"; \ fi ; \ $(distuninstallcheck_listfiles) ; \ exit 1; } >&2 distcleancheck: distclean @if test '$(srcdir)' = . ; then \ echo "ERROR: distcleancheck can only run from a VPATH build" ; \ exit 1 ; \ fi @test `$(distcleancheck_listfiles) | wc -l` -eq 0 \ || { echo "ERROR: files left in build directory after distclean:" ; \ $(distcleancheck_listfiles) ; \ exit 1; } >&2 check-am: all-am check: check-recursive all-am: Makefile $(DATA) $(HEADERS) installdirs: installdirs-recursive installdirs-am: for dir in "$(DESTDIR)$(pkgconfigdir)" "$(DESTDIR)$(includedir)"; do \ test -z "$$dir" || $(MKDIR_P) "$$dir"; \ done install: install-recursive install-exec: install-exec-recursive install-data: install-data-recursive uninstall: uninstall-recursive install-am: all-am @$(MAKE) $(AM_MAKEFLAGS) install-exec-am install-data-am installcheck: installcheck-recursive install-strip: if test -z '$(STRIP)'; then \ $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ install; \ else \ $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ "INSTALL_PROGRAM_ENV=STRIPPROG='$(STRIP)'" install; \ fi mostlyclean-generic: clean-generic: distclean-generic: -test -z "$(CONFIG_CLEAN_FILES)" || rm -f $(CONFIG_CLEAN_FILES) -test . = "$(srcdir)" || test -z "$(CONFIG_CLEAN_VPATH_FILES)" || rm -f $(CONFIG_CLEAN_VPATH_FILES) maintainer-clean-generic: @echo "This command is intended for maintainers to use" @echo "it deletes files that may require special tools to rebuild." clean: clean-recursive clean-am: clean-generic clean-libtool mostlyclean-am distclean: distclean-recursive -rm -f $(am__CONFIG_DISTCLEAN_FILES) -rm -f Makefile distclean-am: clean-am distclean-generic distclean-libtool \ distclean-tags dvi: dvi-recursive dvi-am: html: html-recursive html-am: info: info-recursive info-am: install-data-am: install-nobase_includeHEADERS install-pkgconfigDATA install-dvi: install-dvi-recursive install-dvi-am: install-exec-am: install-html: install-html-recursive install-html-am: install-info: install-info-recursive install-info-am: install-man: install-pdf: install-pdf-recursive install-pdf-am: install-ps: install-ps-recursive install-ps-am: installcheck-am: maintainer-clean: maintainer-clean-recursive -rm -f $(am__CONFIG_DISTCLEAN_FILES) -rm -rf $(top_srcdir)/autom4te.cache -rm -f Makefile maintainer-clean-am: distclean-am maintainer-clean-generic mostlyclean: mostlyclean-recursive mostlyclean-am: mostlyclean-generic mostlyclean-libtool pdf: pdf-recursive pdf-am: ps: ps-recursive ps-am: uninstall-am: uninstall-nobase_includeHEADERS uninstall-pkgconfigDATA .MAKE: $(am__recursive_targets) install-am install-strip .PHONY: $(am__recursive_targets) CTAGS GTAGS TAGS all all-am \ am--refresh check check-am clean clean-cscope clean-generic \ clean-libtool cscope cscopelist-am ctags ctags-am dist \ dist-all dist-bzip2 dist-gzip dist-lzip dist-shar dist-tarZ \ dist-xz dist-zip distcheck distclean distclean-generic \ distclean-libtool distclean-tags distcleancheck distdir \ distuninstallcheck dvi dvi-am html html-am info info-am \ install install-am install-data install-data-am install-dvi \ install-dvi-am install-exec install-exec-am install-html \ install-html-am install-info install-info-am install-man \ install-nobase_includeHEADERS install-pdf install-pdf-am \ install-pkgconfigDATA install-ps install-ps-am install-strip \ installcheck installcheck-am installdirs installdirs-am \ maintainer-clean maintainer-clean-generic mostlyclean \ mostlyclean-generic mostlyclean-libtool pdf pdf-am ps ps-am \ tags tags-am uninstall uninstall-am \ uninstall-nobase_includeHEADERS uninstall-pkgconfigDATA .PRECIOUS: Makefile test: all cd test && ./RunTests -d -f -E 0.001 ../vowpalwabbit/vw ../vowpalwabbit/vw # Tell versions [3.59,3.63) of GNU make to not export all variables. # Otherwise a system limit (for SysV at least) may be exceeded. .NOEXPORT: vowpal-wabbit-8.5.0.dfsg1/R/000077500000000000000000000000001323713302100155015ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/R/example.yaml000066400000000000000000000005531323713302100200230ustar00rootroot00000000000000# YAML FILE ### FORMAT ### #namespaceA: # varName: # - nameVar1 # - nameVar2 # keepSpace: # - false # #namespaceB: # varName: # - nameVar1 # - nameVar2 # keepSpace: # - true sepal: varName: - Sepal.|Length - Sepal.Width keepSpace: - false petal: varName: - Petal.Length - Petal.Width keepSpace: - false vowpal-wabbit-8.5.0.dfsg1/R/examples/000077500000000000000000000000001323713302100173175ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/R/examples/vw_example.R000066400000000000000000000105601323713302100216130ustar00rootroot00000000000000rm(list = ls(all = TRUE)); gc() # setwd('rvw_example') # library(ggplot2) # library(data.table) # library(pROC) library(r.vw) # create a folder called data system('mkdir data') #source('dt2vw.R') #source('rvw_example/vw.R') # Function used to select variables for each namespace get_feature_type <- function(X, threshold = 50, verbose = FALSE) { q_levels <- function (x) { if (data.table::is.data.table(x)) { unlist(x[, lapply(.SD, function(x) length(unique(x)))]) } else { apply(x, 2, function(x) length(unique(x))) } } lvs = q_levels(X) fact_vars = names(lvs[lvs < threshold]) num_vars = names(lvs[lvs >= threshold]) if (verbose) { print(data.frame(lvs)) } list(fact_vars = fact_vars, num_vars = num_vars) } # setwd where the data would be setwd('data') dt = diamonds dt = data.table::setDT(dt) target = 'y' data_types = get_feature_type(dt[, setdiff(names(dt), target), with=F], threshold = 50) namespaces = list(n = list(varName = data_types$num_vars, keepSpace=F), c = list(varName = data_types$fact_vars, keepSpace=F)) dt$y = with(dt, ifelse(y < 5.71, 1, -1)) dt2vw(dt, 'diamonds.vw', namespaces, target=target, weight=NULL) system('head -3 diamonds.vw') # prepare dataset for validation system('head -10000 diamonds.vw > X_train.vw ') system('tail -43940 diamonds.vw > X_valid.vw ') write.table(tail(dt$y,43940), file='valid_labels.txt', row.names = F, col.names = F, quote = F) training_data='X_train.vw' validation_data='X_valid.vw' validation_labels = "valid_labels.txt" out_probs = "out.txt" model = "mdl.vw" # AUC using perf - Download at: osmot.cs.cornell.edu/kddcup/software.html # It may not work, so a dependency of an R library has been added. See below. # Commented as could not work. # auc = vw(training_data, validation_data, loss = "logistic", # model, b = 25, learning_rate = 0.5, passes = 1, # l1 = NULL, l2 = NULL, early_terminate = NULL, # link_function = "--link=logistic", extra = NULL, out_probs = "out.txt", # validation_labels = validation_labels, verbose = TRUE, do_evaluation = TRUE) # Shows files in the working directory: /data list.files() auc = vw(training_data, validation_data, loss = "logistic", model, b = 25, learning_rate = 0.5, passes = 1, l1 = NULL, l2 = NULL, early_terminate = NULL, link_function = "--link=logistic", extra = NULL, out_probs = "out.txt", validation_labels = validation_labels, verbose = TRUE, do_evaluation = TRUE, use_perf=FALSE, plot_roc=TRUE) print(auc) # [1] 0.9944229 # AUC using pROC - Saving plots to disk ### create a parameter grid grid = expand.grid(l1=c(1e-07, 1e-08), l2=c(1e-07, 1e-08), eta=c(0.1, 0.05), extra=c('--nn 10', '')) cat('Running grid search\n') pdf('ROCs.pdf') aucs <- lapply(1:nrow(grid), function(i){ g = grid[i, ] auc = vw(training_data=training_data, # files relative paths validation_data=validation_data, validation_labels=validation_labels, model=model, # grid options loss='logistic', b=25, learning_rate=g[['eta']], passes=2, l1=g[['l1']], l2=g[['l2']], early_terminate=2, extra=g[['extra']], # ROC-AUC related options use_perf=FALSE, plot_roc=TRUE, do_evaluation = TRUE # If false doesn't compute AUC, use only for prediction ) auc }) dev.off() results = cbind(iter=1:nrow(grid), grid, auc=do.call(rbind, aucs)) print(results) # l1 l2 eta extra auc # 1 1e-07 1e-07 0.10 --nn 10 0.9964736 # 2 1e-08 1e-07 0.10 --nn 10 0.9964945 # 3 1e-07 1e-08 0.10 --nn 10 0.9964736 # 4 1e-08 1e-08 0.10 --nn 10 0.9964946 # 5 1e-07 1e-07 0.05 --nn 10 0.9956487 # 6 1e-08 1e-07 0.05 --nn 10 0.9956629 # 7 1e-07 1e-08 0.05 --nn 10 0.9956487 # 8 1e-08 1e-08 0.05 --nn 10 0.9956629 # 9 1e-07 1e-07 0.10 0.9878654 # 10 1e-08 1e-07 0.10 0.9919489 # 11 1e-07 1e-08 0.10 0.9878646 # 12 1e-08 1e-08 0.10 0.9919487 # 13 1e-07 1e-07 0.05 0.9883343 # 14 1e-08 1e-07 0.05 0.9915172 # 15 1e-07 1e-08 0.05 0.9883339 # 16 1e-08 1e-08 0.05 0.9915170 p = ggplot(results, aes(iter, auc, color=extra)) + geom_point(size=3) + theme_bw() + labs(list(x='Iteration', y='AUC', title='Logistic regression results')) print(p) ggsave('results_plot.png', plot=p) vowpal-wabbit-8.5.0.dfsg1/R/examples/vw_example_2.R000066400000000000000000000015101323713302100220270ustar00rootroot00000000000000library(r.vw) ## data data("diamonds", package = "ggplot2") dt = diamonds dt$y = with(dt, ifelse(y < 5.71, 1, -1)) ## separate train and validation data ind_train = sample(1:nrow(dt), 40000) dt_train = dt[ind_train,] dt_val = dt[-ind_train,] ## first method: creating the vw data files before training dt2vw(data = dt_train, fileName = "diamond_train.vw", target = "y") dt2vw(data = dt_val, fileName = "diamond_val.vw", target = "y") write.table(x = dt_val$y, file = "valid_labels.txt", row.names = F, col.names = F) auc1 = vw(training_data = "diamond_train.vw", validation_data = "diamond_val.vw", validation_labels = "valid_labels.txt", use_perf = F) ## 2 method: use directly data.frames auc2 = vw(training_data = dt_train, validation_data = dt_val, target = "y", use_perf = F) vowpal-wabbit-8.5.0.dfsg1/R/r.vw/000077500000000000000000000000001323713302100163755ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/R/r.vw/DESCRIPTION000066400000000000000000000005131323713302100201020ustar00rootroot00000000000000Package: r.vw Type: Package Title: interface between R and Vowpal Wabbit Version: 0.1 Date: 2015-10-07 Authors@R: person("First", "Last", email = "first.last@example.com", role = c("aut", "cre")) Description: More about what it does (maybe more than one line) License: BSD LazyData: TRUE Imports: pROC, data.table, ggplot2 vowpal-wabbit-8.5.0.dfsg1/R/r.vw/NAMESPACE000066400000000000000000000001571323713302100176170ustar00rootroot00000000000000# Generated by roxygen2 (4.1.1): do not edit by hand export(dt2vw) export(vw) import(data.table) import(pROC) vowpal-wabbit-8.5.0.dfsg1/R/r.vw/R/000077500000000000000000000000001323713302100165765ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/R/r.vw/R/dt2vw.R000066400000000000000000000201121323713302100177630ustar00rootroot00000000000000#'Create a vw data file from a R data.frame object #' #'@param data [data.table] data.table format (to be transformed) #'@param fileName [string] file name of the resulting data in VW-friendly format #'@param namespaces [list / yaml file] name of each namespace and each variable for each namespace #'can be a R list, or a YAML file example namespace with the IRIS database: #'namespaces = list(sepal = list(varName = c('Sepal.Length', 'Sepal.Width'), keepSpace=F), #'petal = list(varName = c('Petal.Length', 'Petal.Width'), keepSpace=F)) #'this creates 2 namespaces (sepal and petal) containing the variables defined by varName. #'keepSpace allows to keep or remove spaces in categorical variables #'example: "FERRARI 4Si" ==> "FERRARI_4Si" with keepSpace = F #'==> "FERRARI 4Si" with keepSpace = T (interpreted #'by VW as two distinct categorical variables) #'@param target [string] target of the data (target) #'@param weight [string] weight of each line of the dataset (importance) #'@param tag [string] tag of each line of the dataset #'@param hard_parse [bool] if equals true, parses the data more strictly to avoid feeding VW with false categorical #'variables like '_', or same variables perceived differently like "_var" and "var" #'@import data.table #'@export dt2vw <- function(data, fileName, namespaces = NULL, target, weight = NULL, tag = NULL, hard_parse = FALSE, append = FALSE) { data = setDT(data) #change target if its boolean to take values in {-1,1} if(is.logical(data[[target]]) || sum(levels(factor(data[[target]])) == levels(factor(c(0,1)))) == 2) { data[[target]][data[[target]] == TRUE] = 1 data[[target]][data[[target]] == FALSE] = -1 } #if namespaces = NULL, define a unique namespace if(is.null(namespaces)) { all_vars = colnames(data)[!colnames(data) %in% c(target, weight, tag)] namespaces <- list(A = list(varName = all_vars, keepSpace=FALSE)) } #parse variable names specChar = '\\(|\\)|\\||\\:' specCharSpace = '\\(|\\)|\\||\\:| ' parsingNames <- function(x) Reduce(c, lapply(x, function(X) gsub(specCharSpace,'_', X))) #parse categorical variables parsingVar <- function(x, keepSpace, hard_parse) { #remove leading and trailing spaces, then remove special characters then remove isolated underscores. if(!keepSpace) spch = specCharSpace else spch = specChar if(hard_parse) gsub('(^_( *|_*)+)|(^_$)|(( *|_*)+_$)|( +_+ +)',' ', gsub(specChar,'_', gsub('(^ +)|( +$)', '',x))) else gsub(spch, '_', x) } ### NAMESPACE LOAD WITH A YAML FILE if(typeof(namespaces) == "character" && length(namespaces) == 1 && str_sub(namespaces, -4, -1) == "yaml") { print("############### USING YAML FILE FOR LOADING THE NAMESPACES ###############") library(yaml) namespaces = yaml.load_file(namespaces) } ### AVOIDING DATA FORMAT PROBLEMS setnames(data, names(data), parsingNames(names(data))) names(namespaces) <- parsingNames(names(namespaces)) for(x in names(namespaces)) namespaces[[x]]$varName = parsingNames(namespaces[[x]]$varName) target = parsingNames(target) if(!is.null(tag)) tag = parsingNames(tag) if(!is.null(weight)) weight = parsingNames(weight) ### INITIALIZING THE HEADER AND INDEX #Header: list of variables'name for each namespace #Index: check if the variable is numerical (->TRUE) or categorical (->FALSE) Header = list() Index = list() for(namespaceName in names(namespaces)) { Index[[namespaceName]] = vapply(data[,namespaces[[namespaceName]][['varName']],with=FALSE], is.numeric, logical(1)) #Header[[namespaceName]][Index[[namespaceName]]] = namespaces[[namespaceName]][['varName']][Index[[namespaceName]]] Header[[namespaceName]] = namespaces[[namespaceName]][['varName']] ### ESCAPE THE CATEGORICAL VARIABLES if(namespaces[[namespaceName]]$keepSpace) Header[[namespaceName]][!Index[[namespaceName]]] = paste0("eval(parse(text = 'parsingVar(", Header[[namespaceName]][!Index[[namespaceName]]], ", keepSpace = T, hard_parse = hard_parse)'))") else Header[[namespaceName]][!Index[[namespaceName]]] = paste0("eval(parse(text = 'parsingVar(", Header[[namespaceName]][!Index[[namespaceName]]], ", keepSpace = F, hard_parse = hard_parse)'))") } #appending the name of the variable to its value for each categorical variable sapply(Index, FUN = function(x){sapply(names(x), FUN = function(y){if(!x[[y]]){ set(data, i=NULL, y, paste0(y,"_",data[[y]])) }})}) ### FIRST PART OF THE VW DATA FORMAT: target, weight, tag formatDataVW = '' argexpr = character(0) ### Label can be null, no training is performed if(!is.null(target)) { # Both weight and tag are not null if(!is.null(weight) && !is.null(tag)) { formatDataVW = '%f %f %s' argexpr = paste(target, weight, tag, sep = ', ') } # Weight is null, tag is not null else if(is.null(weight) && !is.null(tag)) { formatDataVW = '%f %s' argexpr = paste(target, tag, sep = ', ') } # Weight is not null, tag is null else if(!is.null(weight) && is.null(tag)) { formatDataVW = '%f %f' argexpr = paste(target, weight, sep = ', ') } # We just output target else { formatDataVW = '%f' argexpr = target } } ### ADDING THE FORMAT FOR THE VARIABLES OF EACH NAMESPACE, AND CREATING THE ARGUMENT VECTOR for(namespaceName in names(namespaces)) { header = Header[[namespaceName]] index = Index[[namespaceName]] formatNumeric = paste0(header[index], rep(":%f ", sum(index)), collapse = "") formatCategorical = paste0(rep("%s", sum(!index)), collapse = " ") formatDataVW = c(formatDataVW, paste0(namespaceName, ' ', formatNumeric, formatCategorical)) paramexpr = paste0(c(header[index], header[!index] ), collapse=', ') argexpr = paste0(c(argexpr, paramexpr), collapse = ', ') } ### FULL VW DATA STRING (NOT FORMATTED YET) : (%target %weight |A num1:%f %s |B num2:%f %s) if (!is.null(tag)) { formatDataVW = paste0(formatDataVW, collapse = '|') } else { formatDataVW = paste0(formatDataVW, collapse = ' |') } formatDataVW = paste0("sprintf2('", formatDataVW, "',",argexpr, ")") ### FORMATTING USING THE DATA.TABLE DYNAMICS TO OBTAIN THE FINAL VW DATA STRING temp = data[, eval(parse(text = formatDataVW))] temp = paste0(temp, collapse = '\n') ### WRITING THE DATA TO A FILE if(!append) con = file(fileName,"w") else con = file(fileName,"a") writeLines(temp,con = con) close(con) } ## Work around the "only 100 arguments are allowed" error ## in base::sprintf(). Only works with 'fmt' of length 1. ## Work around the "only 100 arguments are allowed" error ## in base::sprintf(). Only works with 'fmt' of length 1. sprintf2 <- function(fmt, ...) { MAX_NVAL <- 99L args <- list(...) if (length(args) <= MAX_NVAL) return(sprintf(fmt, ...)) stopifnot(length(fmt) == 1L) not_a_spec_at <- gregexpr("%%", fmt, fixed=TRUE)[[1L]] not_a_spec_at <- c(not_a_spec_at, not_a_spec_at + 1L) spec_at <- setdiff(gregexpr("%", fmt, fixed=TRUE)[[1L]], not_a_spec_at) nspec <- length(spec_at) if (length(args) < nspec) stop("too few arguments") if (nspec <= MAX_NVAL) { break_points <- integer(0) } else { break_points <- seq(MAX_NVAL + 1L, nspec, by=MAX_NVAL) } break_from <- c(1L, break_points) break_to <- c(break_points - 1L, nspec) fmt_break_at <- spec_at[break_points] fmt_chunks <- substr(rep.int(fmt, length(fmt_break_at) + 1L), c(1L, fmt_break_at), c(fmt_break_at - 1L, nchar(fmt))) ans_chunks <- mapply( function(fmt_chunk, from, to) do.call(sprintf, c(list(fmt_chunk), args[from:to])), fmt_chunks, break_from, break_to ) paste(apply(ans_chunks,1, paste, collapse = ""), collapse = "\n") } vowpal-wabbit-8.5.0.dfsg1/R/r.vw/R/vw.R000066400000000000000000000166731323713302100173720ustar00rootroot00000000000000#'Trains Vowpal Wabbit models from R. #' #'This function is fairly simple and extensible to other problems, so far just supports binary classification. #'Thought to be used in conjuction to perf in order to compute validation metrics on left out datasets. #'See osmot.cs.cornell.edu/kddcup/software.html for more info about perf. #' #'@param training_data a [data.frame] or path to a vw data file #'@param validation_data a [data.frame] or path to a vw data file #'@param model name of the model file #'@param path_vw_data_train if training_data is a [data.frame], the path to which to save #'the vw data file. If NULL, the data is stored in a temporary folder and deleted before exiting #'the function #'@param path_vw_data_val if validation_data is a [data.frame], the path to which to save #'the vw data file. If NULL, the data is stored in a temporary folder and deleted before exiting #'the function #'@param target if training_data or validation_data is a [data.frame], the name of the variable #'in the [data.frame] corresponding to the target variable #'@param namespaces used only if training_data or validation_data is a [data.frame]. See arguments #' of dt2vw #'@param weight used only if training_data or validation_data is a [data.frame]. See arguments #' of dt2vw #'@param tag used only if training_data or validation_data is a [data.frame]. See arguments #' of dt2vw #' @param out_probs path to file where to save the predictions. If NULL, the file is stored in #' a temporary file then deleted. #'@param loss loss function. By default logistic. #'@param b number of bits for the weight vector allocation #'@param learning_rate #'@param passes #'@param l1 l1 regularization #'@param l2 l2 regularization #'@param early_terminate #'@param interactions Add interaction terms. Can be passed in extra also. #'@param link_function used to generate predictions #'@param extra These is where more VW commands can be passed as text #'@param out_probs filename to write probabilities #'@param validation_labels file to look for validation data true labels - to compute auc using perf #'or roc_auc() from the R package pROC. If the validation data is a [data.frame] and validation_labels #'is NULL, the validation labels file is deleted before exiting the function. If validation_labels is not #'NULL, it indicates the path where validation labels should be stored. #'@param verbose mostly used to debug but shows AUC and the vw command used to train the model #'@param keep_preds TRUE (default) to return a vector of the predictions #'@param do_evaluation TRUE to compute auc on validation_data. Use FALSE, to just score data #'@param use_perf use perf to compute auc. Otherwise, auc_roc() from the R package pROC is used. #'@examples #'# 1. Create a training set (training_data) and validation set (validation_data) in vw format. #'# 2. Install perf #'# 3. Create a vector of true labels for the validation dataset, in the [0, 1] range. This is what perf likes. #'# 4. Run one model with the present code #' #'\dontrun{ #' auc = vw(training_data='X_train.vw', validation_data='X_valid.vw', #' loss='logistic', model='mdl.vw', b=25, learning_rate=0.5, #' passes=20, l1=1e-08, l2=1e-08, early_terminate=2, #' interactions=NULL, extra='--stage_poly') #'} #'@import pROC #'@export vw <- function(training_data, validation_data, model='mdl.vw', path_vw_data_train = NULL, path_vw_data_val = NULL, target = NULL, namespaces = NULL, weight = NULL, tag = NULL, out_probs= NULL, validation_labels= NULL, loss='logistic', b=25, learning_rate=0.5, passes=1, l1=NULL, l2=NULL, early_terminate=NULL, link_function='--link=logistic', extra=NULL, keep_preds = TRUE, do_evaluation=TRUE, use_perf=TRUE, plot_roc=TRUE, verbose=TRUE){ ## this should not create an unnecessary copy of the arguments data_args = list(train = training_data, val = validation_data) path_data_args = list(path_vw_data_train, path_vw_data_val) for(i in seq_along(data_args)) { if (inherits(data_args[[i]], "data.frame")) { if(is.null(target)) stop(paste0(names(data_args)[i], "data argument: input argument is a data.frame, argument 'target' should be specified ")) if(! is.character(path_data_args[[i]])) path_data_args[[i]] = file.path(tempdir(),paste0(names(data_args)[i],".vw")) dt2vw(data = data_args[[i]], fileName = path_data_args[[i]], namespaces = namespaces, target = target, weight = weight, tag = tag) } else { path_data_args[[i]] = data_args[[i]] } } training_data = path_data_args[[1]] cmd = sprintf('vw -d %s --loss_function %s -f %s', training_data, loss, model) cmd = sprintf('%s --learning_rate=%s --passes %s -c', cmd, learning_rate, passes) if(!is.null(l1)) cmd = sprintf('%s --l1 %s', cmd, l1) if(!is.null(l2)) cmd = sprintf('%s --l2 %s', cmd, l2) if(!is.null(b)) cmd = sprintf('%s -b %s', cmd, b) if(!is.null(early_terminate)) cmd = sprintf('%s --early_terminate %s', cmd, early_terminate) if(!is.null(extra)) cmd = sprintf('%s %s', cmd, extra) cat('Model parameters\n') cat(cmd) cat('\n') system(cmd) if(is.null(out_probs)) { out_probs = file.path(tempdir(),"preds.vw") del_prob = TRUE } else del_prob = FALSE validation_data = path_data_args[[2]] predict = sprintf('vw -t -i %s -p %s %s -d %s', model, out_probs, link_function, validation_data) system(predict) if(do_evaluation){ if(inherits(data_args[[2]], "data.frame")) { if(is.null(validation_labels)) { del_val = TRUE validation_labels = file.path(tempdir(),"val_labs.vw") } else del_val = FALSE write.table(x = data_args[[2]][[target]], file = validation_labels, row.names = FALSE, col.names = FALSE) } if(use_perf){ # compute auc using perf eval_model = sprintf("perf -ROC -files %s %s | cut -c8-14", validation_labels, out_probs) auc = system(eval_model, intern = TRUE) } else { auc = roc_auc(out_probs, validation_labels, plot_roc, cmd) } } if(verbose && do_evaluation){ cat('Model Parameters\n') cat(cmd) verbose_log = sprintf('AUC: %s', auc) print(verbose_log) } if(keep_preds) probs = fread(out_probs)[['V1']] ## delete temporary files for(i in seq_along(data_args)) if(inherits(data_args[[i]], "data.frame")) file.remove(path_data_args[[i]]) if(del_prob) file.remove(out_probs) if(exists("del_val") && del_val) file.remove(validation_labels) return(list(auc=auc, preds=probs)) } # Reads labels file (from the validation dataset) # and probabilities (out_file) from vowpal wabbit # Also added an option to plot or not the AUC roc_auc <- function(out_probs, validation_labels, plot_roc, cmd, ...){ probs = fread(out_probs)[['V1']] labels = fread(validation_labels)[['V1']] if(!identical(length(probs), length(labels))) stop('The length of the probabilities and labels is different') # Fix cmd for adding it in title cmd = vapply(strsplit(cmd, '-f'), function(x) paste0(x, collapse='\n'), character(1)) cmd = vapply(strsplit(cmd, '-c'), function(x) paste0(x, collapse='\n'), character(1)) # Plot ROC curve and return AUC roc = roc(labels, probs, auc=TRUE, print.auc=TRUE, print.thres=TRUE) if(plot_roc){ print(plot.roc(roc, main=cmd, cex.main = 0.5, ...)) } auc_value = as.numeric(roc$auc[[1]]) return(auc_value) } vowpal-wabbit-8.5.0.dfsg1/R/r.vw/man/000077500000000000000000000000001323713302100171505ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/R/r.vw/man/dt2vw.Rd000066400000000000000000000030061323713302100205040ustar00rootroot00000000000000% Generated by roxygen2 (4.1.1): do not edit by hand % Please edit documentation in R/dt2vw.R \name{dt2vw} \alias{dt2vw} \title{Create a vw data file from a R data.frame object} \usage{ dt2vw(data, fileName, namespaces = NULL, target, weight = NULL, tag = NULL, hard_parse = F, append = F) } \arguments{ \item{data}{[data.table] data.table format (to be transformed)} \item{fileName}{[string] file name of the resulting data in VW-friendly format} \item{namespaces}{[list / yaml file] name of each namespace and each variable for each namespace can be a R list, or a YAML file example namespace with the IRIS database: namespaces = list(sepal = list(varName = c('Sepal.Length', 'Sepal.Width'), keepSpace=F), petal = list(varName = c('Petal.Length', 'Petal.Width'), keepSpace=F)) this creates 2 namespaces (sepal and petal) containing the variables defined by varName. keepSpace allows to keep or remove spaces in categorical variables example: "FERRARI 4Si" ==> "FERRARI_4Si" with keepSpace = F ==> "FERRARI 4Si" with keepSpace = T (interpreted by VW as two distinct categorical variables)} \item{target}{[string] target of the data (target)} \item{weight}{[string] weight of each line of the dataset (importance)} \item{tag}{[string] tag of each line of the dataset} \item{hard_parse}{[bool] if equals true, parses the data more strictly to avoid feeding VW with false categorical variables like '_', or same variables perceived differently like "_var" and "var"} } \description{ Create a vw data file from a R data.frame object } vowpal-wabbit-8.5.0.dfsg1/R/r.vw/man/vw.Rd000066400000000000000000000073651323713302100201060ustar00rootroot00000000000000% Generated by roxygen2 (4.1.1): do not edit by hand % Please edit documentation in R/vw.R \name{vw} \alias{vw} \title{Trains Vowpal Wabbit models from R.} \usage{ vw(training_data, validation_data, model = "mdl.vw", path_vw_data_train = NULL, path_vw_data_val = NULL, target = NULL, namespaces = NULL, weight = NULL, tag = NULL, out_probs = NULL, validation_labels = NULL, loss = "logistic", b = 25, learning_rate = 0.5, passes = 1, l1 = NULL, l2 = NULL, early_terminate = NULL, link_function = "--link=logistic", extra = NULL, do_evaluation = TRUE, use_perf = TRUE, plot_roc = TRUE, verbose = TRUE) } \arguments{ \item{training_data}{a [data.frame] or path to a vw data file} \item{validation_data}{a [data.frame] or path to a vw data file} \item{model}{name of the model file} \item{path_vw_data_train}{if training_data is a [data.frame], the path to which to save the vw data file. If NULL, the data is stored in a temporary folder and deleted before exiting the function} \item{path_vw_data_val}{if validation_data is a [data.frame], the path to which to save the vw data file. If NULL, the data is stored in a temporary folder and deleted before exiting the function} \item{target}{if training_data or validation_data is a [data.frame], the name of the variable in the [data.frame] corresponding to the target variable} \item{namespaces}{used only if training_data or validation_data is a [data.frame]. See arguments of dt2vw} \item{weight}{used only if training_data or validation_data is a [data.frame]. See arguments of dt2vw} \item{tag}{used only if training_data or validation_data is a [data.frame]. See arguments of dt2vw} \item{out_probs}{path to file where to save the predictions. If NULL, the file is stored in a temporary file then deleted.} \item{validation_labels}{file to look for validation data true labels - to compute auc using perf or roc_auc() from the R package pROC. If the validation data is a [data.frame] and validation_labels is NULL, the validation labels file is deleted before exiting the function. If validation_labels is not NULL, it indicates the path where validation labels should be stored.} \item{loss}{loss function. By default logistic.} \item{b}{number of bits for the weight vector allocation} \item{learning_rate}{} \item{passes}{} \item{l1}{l1 regularization} \item{l2}{l2 regularization} \item{early_terminate}{} \item{link_function}{used to generate predictions} \item{extra}{These is where more VW commands can be passed as text} \item{keep_preds}{TRUE (default) to return a vector of the predictions} \item{do_evaluation}{TRUE to compute auc on validation_data. Use FALSE, to just score data} \item{use_perf}{use perf to compute auc. Otherwise, auc_roc() from the R package pROC is used.} \item{verbose}{mostly used to debug but shows AUC and the vw command used to train the model} \item{interactions}{Add interaction terms. Can be passed in extra also.} \item{out_probs}{filename to write probabilities} } \description{ This function is fairly simple and extensible to other problems, so far just supports binary classification. Thought to be used in conjuction to perf in order to compute validation metrics on left out datasets. See osmot.cs.cornell.edu/kddcup/software.html for more info about perf. } \examples{ # 1. Create a training set (training_data) and validation set (validation_data) in vw format. # 2. Install perf # 3. Create a vector of true labels for the validation dataset, in the [0, 1] range. This is what perf likes. # 4. Run one model with the present code \dontrun{ auc = vw(training_data='X_train.vw', validation_data='X_valid.vw', loss='logistic', model='mdl.vw', b=25, learning_rate=0.5, passes=20, l1=1e-08, l2=1e-08, early_terminate=2, interactions=NULL, extra='--stage_poly') } } vowpal-wabbit-8.5.0.dfsg1/R/rscripts_readme.md000066400000000000000000000015631323713302100212160ustar00rootroot00000000000000### R scripts for vowpal wabbit ### download the r.vw package r.vw contains two functions: dt2vw and vw installation requires devtools: ``` install.packages("devtools") devtools::install_github("JohnLangford/vowpal_wabbit", subdir = "R/r.vw") ``` ### Convert a data.table to vowpal wabbit format: dt2vw() Allows to convert the data.table in chunks using the append=TRUE option. Make sure to define the correct data type before using the function. The function handles different data types as expected from R, so these should be defined already in the data.table object. ### Call vowpal wabbit from R: vw() Follow the example in R/examples/vw_example.R and R/examples/vw_example_2.R. It uses the vw.R function to run VW using system commands, so it is simple to adapt to different models. It also computes the AUC on a validation test set and plots the ROC curve if needed. vowpal-wabbit-8.5.0.dfsg1/README.deploy.txt000066400000000000000000000031411323713302100202700ustar00rootroot00000000000000Deploying Vowpal Wabbit Nick Nussbaum 9/7/14 The deploy_vw project makes a folder containing vw.exe,spanning tree.exe together with the the Redistributable Visual Studio 2013 c++ runtime. You can put the folder on a machine which does not have Visual Studio 2013 installed and run the vw.exe without installing anything. If your environment has Visual Studio 2013 installed, there's no point in doing this since the runtime files are already installed on the system. Th deploy_vw projects are by default unchecked in the build configuration manager so they aren't built everytime the solution is compiled. Normally you'd give people the x86 or x64 release versions of the deployment folder If you're debugging clusters with a remote debugger, you may want to build the debug versions. These includes the pdb files as well as the debug versions of the runtime. These debug runtime versions are not redistributable. You can use them in your machines that have a Visual Studio Licence but can't distribute them to machines without a Visual Studio license. Check the Microsoft documentation for details on this restriction. To make the deployment In the Build > Configuration Manager menu set the Active Configuration and Active Platform. X64 and release would be a common choice. Select thed deploy_vw project in Solution Explorer select Build Menu< Clean deploy_vw select Build Menu, Rebuild deploy_vw This will trigger builds of vw and spanning tree if needed and then create the folder vowpal_wabbit\vowpalwabbit\deploy\x64\Release. This folder can be copied to other Windows machines in order to run Vowpal Wabbit. vowpal-wabbit-8.5.0.dfsg1/README.md000066400000000000000000000162241323713302100165640ustar00rootroot00000000000000``` /* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ ``` Vowpal Wabbit [![Build Status](https://travis-ci.org/JohnLangford/vowpal_wabbit.png)](https://travis-ci.org/JohnLangford/vowpal_wabbit) [![Windows Build Status](https://ci.appveyor.com/api/projects/status/github/JohnLangford/vowpal_wabbit?branch=master&svg=true)](https://ci.appveyor.com/project/JohnLangford/vowpal-wabbit) [![Coverage Status](https://coveralls.io/repos/JohnLangford/vowpal_wabbit/badge.svg)](https://coveralls.io/r/JohnLangford/vowpal_wabbit) This is the *vowpal wabbit* fast online learning code. For Windows, look at README.windows.txt ## Prerequisite software These prerequisites are usually pre-installed on many platforms. However, you may need to consult your favorite package manager (*yum*, *apt*, *MacPorts*, *brew*, ...) to install missing software. - [Boost](http://www.boost.org) library, with the `Boost::Program_Options` library option enabled. - The zlib compression library + headers. In linux distros: package `zlib-devel` (Red Hat/CentOS), or `zlib1g-dev` (Ubuntu/Debian) - lsb-release (RedHat/CentOS: redhat-lsb-core, Debian: lsb-release, Ubuntu: you're all set, OSX: not required) - GNU *autotools*: *autoconf*, *automake*, *libtool*, *autoheader*, et. al. This is not a strict prereq. On many systems (notably Ubuntu with `libboost-program-options-dev` installed), the provided `Makefile` works fine. - (optional) [git](http://git-scm.com) if you want to check out the latest version of *vowpal wabbit*, work on the code, or even contribute code to the main project. ## Getting the code You can download the latest version from [here](https://github.com/JohnLangford/vowpal_wabbit/wiki/Download). The very latest version is always available via 'github' by invoking one of the following: ``` ## For the traditional ssh-based Git interaction: $ git clone git://github.com/JohnLangford/vowpal_wabbit.git ## You can also try the following SSH URL: $ git clone git@github.com:JohnLangford/vowpal_wabbit.git ## For HTTP-based Git interaction $ git clone https://github.com/JohnLangford/vowpal_wabbit.git ``` ## Compiling You should be able to build the *vowpal wabbit* on most systems with: ``` $ make $ make test # (optional) ``` If that fails, try: ``` $ ./autogen.sh $ make $ make test # (optional) $ make install ``` Note that `./autogen.sh` requires *automake* (see the prerequisites, above.) `./autogen.sh`'s command line arguments are passed directly to `configure` as if they were `configure` arguments and flags. Note that `./autogen.sh` will overwrite the supplied `Makefile`, including the `Makefile`s in sub-directories, so keeping a copy of the `Makefile`s may be a good idea before running `autogen.sh`. If your original `Makefile`s were overwritten by `autogen.sh` calling `automake`, you may always get the originals back from git using: ``` git checkout Makefile */Makefile ``` Be sure to read the wiki: https://github.com/JohnLangford/vowpal_wabbit/wiki for the tutorial, command line options, etc. The 'cluster' directory has it's own documentation for cluster parallel use, and the examples at the end of test/Runtests give some example flags. ## C++ Optimization The default C++ compiler optimization flags are very aggressive. If you should run into a problem, consider creating and running `configure` with the `--enable-debug` option, e.g.: ``` $ ./configure --enable-debug ``` or passing your own compiler flags via the `OPTIM_FLAGS` make variable: ``` $ make OPTIM_FLAGS="-O0 -g" ``` ## Ubuntu/Debian specific info On Ubuntu/Debian/Mint and similar the following sequence should work for building the latest from github: ``` # -- Get libboost program-options and zlib: apt-get install libboost-program-options-dev zlib1g-dev # -- Get the python libboost bindings (python subdir) - optional: apt-get install libboost-python-dev # -- Get the vw source: git clone git://github.com/JohnLangford/vowpal_wabbit.git # -- Build: cd vowpal_wabbit make make test # (optional) make install ``` ### Ubuntu advanced build options (clang and static) If you prefer building with `clang` instead of `gcc` (much faster build and slighly faster executable), install `clang` and change the `make` step slightly: ``` apt-get install clang make CXX=clang++ ``` A statically linked `vw` executable that is not sensitive to boost version upgrades and can be safely copied between different Linux versions (e.g. even from Ubuntu to Red-Hat) can be built and tested with: ``` make CXX='clang++ -static' clean vw test # ignore warnings ``` ## Mac OS X-specific info OSX requires _glibtools_, which is available via the [brew](http://brew.sh) or [MacPorts](https://www.macports.org) package managers. ### Complete brew install of 8.4 ``` brew install vowpal-wabbit ``` [The homebrew formula for VW is located on github](https://github.com/Homebrew/homebrew-core/blob/master/Formula/vowpal-wabbit.rb). ### Manual install of Vowpal Wabbit #### OSX Dependencies (if using Brew): ``` brew install libtool brew install autoconf brew install automake brew install boost brew install boost-python ``` #### OSX Dependencies (if using MacPorts): ``` ## Install glibtool and other GNU autotool friends: $ port install libtool autoconf automake ## Build Boost for Mac OS X 10.8 and below $ port install boost +no_single +no_static +openmpi +python27 configure.cxx_stdlib=libc++ configure.cxx=clang++ ## Build Boost for Mac OS X 10.9 and above $ port install boost +no_single +no_static +openmpi +python27 ``` #### OSX Manual compile: *Mac OS X 10.8 and below*: ``configure.cxx_stdlib=libc++`` and ``configure.cxx=clang++`` ensure that ``clang++`` uses the correct C++11 functionality while building Boost. Ordinarily, ``clang++`` relies on the older GNU ``g++`` 4.2 series header files and ``stdc++`` library; ``libc++`` is the ``clang`` replacement that provides newer C++11 functionality. If these flags aren't present, you will likely encounter compilation errors when compiling _vowpalwabbit/cbify.cc_. These error messages generally contain complaints about ``std::to_string`` and ``std::unique_ptr`` types missing. To compile: ``` $ sh autogen.sh --enable-libc++ $ make $ make test # (optional) ``` #### OSX Python Binding installation with Anaconda When using Anaconda as the source for Python the default Boost libraries used in the Makefile need to be adjusted. Below are the steps needed to install the Python bindings for VW. This should work for Python 2 and 3. Adjust the directories to match where anaconda is installed. ``` # create anaconda environment with boost conda create --name vw boost source activate vw git clone https://github.com/JohnLangford/vowpal_wabbit.git cd vowpal_wabbit # edit Makefile # change BOOST_INCLUDE to use anaconda env dir: /anaconda/envs/vw/include # change BOOST_LIBRARY to use anaconda lib dir: /andaconda/envs/vw/lib cd python python setup.py install ``` ## Code Documentation To browse the code more easily, do `make doc` and then point your browser to `doc/html/index.html`. vowpal-wabbit-8.5.0.dfsg1/README.windows.txt000066400000000000000000000160001323713302100204640ustar00rootroot00000000000000Originally by Chris Quirk ************************************************************************************************************** Notes for building VW under Visual Studio 2013 on Windows 8.1 9/02/2014 Nick Nussbaum nickn@seanet.com Replace source dependencies with Nuget 04/29/2015 Sharat Chikkerur sharat.chikkerur@gmail.com Added ANTLR based unit test 10/2/2015 Markus Cozowicz marcozo@microsoft.com ************************************************************************************************************** (1) Get Tools You'll need a Visual Studio 2013 (or 2015) installed that includes c# and c++ You should install Visual Studio 2013 Update 5: https://www.microsoft.com/en-us/download/details.aspx?id=48129 You'll also need the Windows SDK which you can download from Microsoft at http://msdn.microsoft.com/en-us/windows/desktop/bg162891.aspx You'll need Nuget integration with visual studio http://docs.nuget.org/consume You'll need Java to run unit tests http://www.oracle.com/technetwork/java/javase/downloads/jre8-downloads-2133155.html ************************************************************************************************************** (2) open a copy various command shells (a) Open an x86 command shell: run the Visual Studio 2013 Tools / VS2013 x86 Native Tools Command Prompt or cmd.exe /k "C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\vcvarsall.bat" x86 (b) Open an x64 command shell: run the Visual Studio 2013 Tools / VS2013 x64 Cross Tools Command Prompt or cmd.exe /k "C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\vcvarsall.bat" x86_amd64 (c) Open the Git bash shell "C:\Program Files (x86)\Git\bin\sh.exe" --login -i or some other bash shell ************************************************************************************************************** (3) Setup Directories I use c:\src\vw as my %ROOT% directory; (a) mkdir c:\src (b) mkdir c:\src\vw ************************************************************************************************************** (4) Get Vowpal Wabbit (a) In a command shell to %ROOT% : "cd c:\src\vw" (b) run "git clone https://github.com/JohnLangford/vowpal_wabbit.git" details of the changes are in bottom of this file. ************************************************************************************************************** (5) Restore nugets (a) In a command shell to %ROOT%\vowpalwabbit\vowpalwabbit : "cd c:\src\vw\vowpalwabbit\vowpalwabbit" (b) run ".nuget\nuget restore vw.sln" This will restore the ANTLR nuget which is needed before Visual Studio loads the solution. ************************************************************************************************************** (5) Build Vowpal Wabbit (a) Using visual studio Open %ROOT%\vowpal_wabbit\vowpalwabbit\vw.sln in Visual Studio 2013 Set startup project as vw (or the test project) Select x64 platform (Configuration Manager \ Active solution platfrom) Select x64 as test platform (Test \ Test settings \ Default Processor Architecture) run build>rebuild solution or run batch build Binaries will be in one of these four directories, based on whether you built DEBUG or RELEASE bits and whether you are building x64. %ROOT%\vowpal_wabbit\vowpalwabbit\x64\Debug\vw.exe %ROOT%\vowpal_wabbit\vowpalwabbit\x64\Release\vw.exe Missing nugets will be installed during the build. (b) Using command line (available configurations are "Release" and "Debug". Available platforms are "x64" and "Win32") run>msbuild /p:Configuration="Release" /p:Platform="x64" vw.sln Note: If you failed to do so before opening the solution, the cs_unittest project is in a "not loaded" state. After executing the above you'll have to hit "Reload" (Project / Context Menu) in Visual Studio. ************************************************************************************************************** (8) Test There's a new test batch file that runs a quick test on all four configurations (a) go to a windows command shell (a) cd c:\src\vw\test (b) run test\test_2_winvw.bat ************************************************************************************************************** (9) Appendix: The Gory Details of the patch and VW upgrades (a) misc files adds this content to this file ReadMe.Windows.txt adds the file vowpal_wabbit\zlibpatch.txt a patch for xlib adds the file test\test_2_winvw.bat a simple test of x86 and x64 training and prediction (b) Changes to Zlib This Zlib patch includes the following fixes; Convert to Visual Studio 2013 solution The fix in the prior section to correctly use DLL versions of the runtime for 32bit platforms Changes to use only two fields in zlibvc.def VERSTION - VERSION 1.2.8 + VERSION 1.28 since otherwise the compiler will complain about more than 2 fields and ignore them. add /safeseh to the x86 assembler so Visual Studio will not generate an error "unable to generate SAFESSH image" This is not need for x64 since it happens by default In the properties sheet for zlibvc The pre build command line for x64 release should be fixed -cd ..\..\contrib\masmx64 +cd ..\..\masmx64 Code generation: Runtime Library for windows release set to Multi-threaded DLL (/MD) not /MT for zlibvc and zlibstat Otherwise VS13 will complain about multiple runtime specification while trying to autolink (c) Change to Boost 1.56.0 (d) Changes to VowpalWabbit changes vw projects and solutions to run under Visual Studio 2013 rather than Visual Studio 2012 change vw projects to redefine $(BoostIncludeDir) to refer to Boost 1.56.0 change vw projects to define $(BoostLibDir) to refer to Boost 1.56.0 vowpalwabbit/vw_static.vcxproj Define $(IncludePath) change $(ZlibDir) to use \contrib\vstudio\vc11 rather than vc10 change x64 version DebugInformationFormat to use "ProgramDatabase" and not the invalid "EditAndContinue" change IntermediateFolderPath to include ProjectName so two projects aren't trying to build in the same folder add searn_multiclasstask.cc to the project change include path to all use macros $(VC_IncludePath);$(WindowsSDK_IncludePath) change additional dependencies to use $(SolutionDir)$(PlatformShortName)\$(Configuration)\vw_static.lib adds a reference to the WindowsSDKDir Include\um change vw_static properties for debug 64bit to /Zi from /Zl to shut up some warnings. change the vw and static_vw to use n intermediate directories that appends the $(ProjectName). this avoid various conflicts and warnings caused by dumping into the same directory. change link build copies to use PlatformShortName rather than PlatformName to use x86 rather than Win32 Change the anycpu confuuration for problems with cs_test vowpalwabbit/vw.sln change configurations to use Debug|x86 from Debug|AnyCpu c_test/c_test.vcxproj change to VS 12 change configurations to use Debug|x86 from Debug|AnyCpu change cs_test to use x86 and x64 rather than anycpu change test file specs to reference the .../../... test directory vowpal-wabbit-8.5.0.dfsg1/acinclude.d/000077500000000000000000000000001323713302100174515ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/acinclude.d/ax_boost_base.m4000066400000000000000000000241531323713302100225300ustar00rootroot00000000000000# =========================================================================== # http://www.gnu.org/software/autoconf-archive/ax_boost_base.html # =========================================================================== # # SYNOPSIS # # AX_BOOST_BASE([MINIMUM-VERSION], [ACTION-IF-FOUND], [ACTION-IF-NOT-FOUND]) # # DESCRIPTION # # Test for the Boost C++ libraries of a particular version (or newer) # # If no path to the installed boost library is given the macro searchs # under /usr, /usr/local, /opt and /opt/local and evaluates the # $BOOST_ROOT environment variable. Further documentation is available at # . # # This macro calls: # # AC_SUBST(BOOST_CPPFLAGS) / AC_SUBST(BOOST_LDFLAGS) # # And sets: # # HAVE_BOOST # # LICENSE # # Copyright (c) 2008 Thomas Porschberg # Copyright (c) 2009 Peter Adolphs # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice # and this notice are preserved. This file is offered as-is, without any # warranty. #serial 20 AC_DEFUN([AX_BOOST_BASE], [ AC_ARG_WITH([boost], [AS_HELP_STRING([--with-boost@<:@=ARG@:>@], [use Boost library from a standard location (ARG=yes), from the specified location (ARG=), or disable it (ARG=no) @<:@ARG=yes@:>@ ])], [ if test "$withval" = "no"; then want_boost="no" elif test "$withval" = "yes"; then want_boost="yes" ac_boost_path="" else want_boost="yes" ac_boost_path="$withval" fi ], [want_boost="yes"]) AC_ARG_WITH([boost-libdir], AS_HELP_STRING([--with-boost-libdir=LIB_DIR], [Force given directory for boost libraries. Note that this will override library path detection, so use this parameter only if default library detection fails and you know exactly where your boost libraries are located.]), [ if test -d "$withval" then ac_boost_lib_path="$withval" else AC_MSG_ERROR(--with-boost-libdir expected directory name) fi ], [ac_boost_lib_path=""] ) if test "x$want_boost" = "xyes"; then boost_lib_version_req=ifelse([$1], ,1.20.0,$1) boost_lib_version_req_shorten=`expr $boost_lib_version_req : '\([[0-9]]*\.[[0-9]]*\)'` boost_lib_version_req_major=`expr $boost_lib_version_req : '\([[0-9]]*\)'` boost_lib_version_req_minor=`expr $boost_lib_version_req : '[[0-9]]*\.\([[0-9]]*\)'` boost_lib_version_req_sub_minor=`expr $boost_lib_version_req : '[[0-9]]*\.[[0-9]]*\.\([[0-9]]*\)'` if test "x$boost_lib_version_req_sub_minor" = "x" ; then boost_lib_version_req_sub_minor="0" fi WANT_BOOST_VERSION=`expr $boost_lib_version_req_major \* 100000 \+ $boost_lib_version_req_minor \* 100 \+ $boost_lib_version_req_sub_minor` AC_MSG_CHECKING(for boostlib >= $boost_lib_version_req) succeeded=no dnl On 64-bit systems check for system libraries in both lib64 and lib. dnl The former is specified by FHS, but e.g. Debian does not adhere to dnl this (as it rises problems for generic multi-arch support). dnl The last entry in the list is chosen by default when no libraries dnl are found, e.g. when only header-only libraries are installed! libsubdirs="lib" ax_arch=`uname -m` if test $ax_arch = x86_64 -o $ax_arch = ppc64 -o $ax_arch = s390x -o $ax_arch = sparc64; then libsubdirs="lib64 lib lib64" fi dnl first we check the system location for boost libraries dnl this location ist chosen if boost libraries are installed with the --layout=system option dnl or if you install boost with RPM if test "$ac_boost_path" != ""; then BOOST_CPPFLAGS="-I$ac_boost_path/include" for ac_boost_path_tmp in $libsubdirs; do if test -d "$ac_boost_path"/"$ac_boost_path_tmp" ; then BOOST_LDFLAGS="-L$ac_boost_path/$ac_boost_path_tmp" break fi done elif test "$cross_compiling" != yes; then for ac_boost_path_tmp in /usr /usr/local /opt /opt/local ; do if test -d "$ac_boost_path_tmp/include/boost" && test -r "$ac_boost_path_tmp/include/boost"; then for libsubdir in $libsubdirs ; do if ls "$ac_boost_path_tmp/$libsubdir/libboost_"* >/dev/null 2>&1 ; then break; fi done BOOST_LDFLAGS="-L$ac_boost_path_tmp/$libsubdir" BOOST_CPPFLAGS="-I$ac_boost_path_tmp/include" break; fi done fi dnl overwrite ld flags if we have required special directory with dnl --with-boost-libdir parameter if test "$ac_boost_lib_path" != ""; then BOOST_LDFLAGS="-L$ac_boost_lib_path" fi CPPFLAGS_SAVED="$CPPFLAGS" CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS" export CPPFLAGS LDFLAGS_SAVED="$LDFLAGS" LDFLAGS="$LDFLAGS $BOOST_LDFLAGS" export LDFLAGS AC_REQUIRE([AC_PROG_CXX]) AC_LANG_PUSH(C++) AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ @%:@include ]], [[ #if BOOST_VERSION >= $WANT_BOOST_VERSION // Everything is okay #else # error Boost version is too old #endif ]])],[ AC_MSG_RESULT(yes) succeeded=yes found_system=yes ],[ ]) AC_LANG_POP([C++]) dnl if we found no boost with system layout we search for boost libraries dnl built and installed without the --layout=system option or for a staged(not installed) version if test "x$succeeded" != "xyes"; then _version=0 if test "$ac_boost_path" != ""; then if test -d "$ac_boost_path" && test -r "$ac_boost_path"; then for i in `ls -d $ac_boost_path/include/boost-* 2>/dev/null`; do _version_tmp=`echo $i | sed "s#$ac_boost_path##" | sed 's/\/include\/boost-//' | sed 's/_/./'` V_CHECK=`expr $_version_tmp \> $_version` if test "$V_CHECK" = "1" ; then _version=$_version_tmp fi VERSION_UNDERSCORE=`echo $_version | sed 's/\./_/'` BOOST_CPPFLAGS="-I$ac_boost_path/include/boost-$VERSION_UNDERSCORE" done fi else if test "$cross_compiling" != yes; then for ac_boost_path in /usr /usr/local /opt /opt/local ; do if test -d "$ac_boost_path" && test -r "$ac_boost_path"; then for i in `ls -d $ac_boost_path/include/boost-* 2>/dev/null`; do _version_tmp=`echo $i | sed "s#$ac_boost_path##" | sed 's/\/include\/boost-//' | sed 's/_/./'` V_CHECK=`expr $_version_tmp \> $_version` if test "$V_CHECK" = "1" ; then _version=$_version_tmp best_path=$ac_boost_path fi done fi done VERSION_UNDERSCORE=`echo $_version | sed 's/\./_/'` BOOST_CPPFLAGS="-I$best_path/include/boost-$VERSION_UNDERSCORE" if test "$ac_boost_lib_path" = ""; then for libsubdir in $libsubdirs ; do if ls "$best_path/$libsubdir/libboost_"* >/dev/null 2>&1 ; then break; fi done BOOST_LDFLAGS="-L$best_path/$libsubdir" fi fi if test "x$BOOST_ROOT" != "x"; then for libsubdir in $libsubdirs ; do if ls "$BOOST_ROOT/stage/$libsubdir/libboost_"* >/dev/null 2>&1 ; then break; fi done if test -d "$BOOST_ROOT" && test -r "$BOOST_ROOT" && test -d "$BOOST_ROOT/stage/$libsubdir" && test -r "$BOOST_ROOT/stage/$libsubdir"; then version_dir=`expr //$BOOST_ROOT : '.*/\(.*\)'` stage_version=`echo $version_dir | sed 's/boost_//' | sed 's/_/./g'` stage_version_shorten=`expr $stage_version : '\([[0-9]]*\.[[0-9]]*\)'` V_CHECK=`expr $stage_version_shorten \>\= $_version` if test "$V_CHECK" = "1" -a "$ac_boost_lib_path" = "" ; then AC_MSG_NOTICE(We will use a staged boost library from $BOOST_ROOT) BOOST_CPPFLAGS="-I$BOOST_ROOT" BOOST_LDFLAGS="-L$BOOST_ROOT/stage/$libsubdir" fi fi fi fi CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS" export CPPFLAGS LDFLAGS="$LDFLAGS $BOOST_LDFLAGS" export LDFLAGS AC_LANG_PUSH(C++) AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ @%:@include ]], [[ #if BOOST_VERSION >= $WANT_BOOST_VERSION // Everything is okay #else # error Boost version is too old #endif ]])],[ AC_MSG_RESULT(yes) succeeded=yes found_system=yes ],[ ]) AC_LANG_POP([C++]) fi if test "$succeeded" != "yes" ; then if test "$_version" = "0" ; then AC_MSG_NOTICE([[We could not detect the boost libraries (version $boost_lib_version_req_shorten or higher). If you have a staged boost library (still not installed) please specify \$BOOST_ROOT in your environment and do not give a PATH to --with-boost option. If you are sure you have boost installed, then check your version number looking in . See http://randspringer.de/boost for more documentation.]]) else AC_MSG_NOTICE([Your boost libraries seems to old (version $_version).]) fi # execute ACTION-IF-NOT-FOUND (if present): ifelse([$3], , :, [$3]) else AC_SUBST(BOOST_CPPFLAGS) AC_SUBST(BOOST_LDFLAGS) AC_DEFINE(HAVE_BOOST,,[define if the Boost library is available]) # execute ACTION-IF-FOUND (if present): ifelse([$2], , :, [$2]) fi CPPFLAGS="$CPPFLAGS_SAVED" LDFLAGS="$LDFLAGS_SAVED" fi ]) vowpal-wabbit-8.5.0.dfsg1/acinclude.d/ax_boost_program_options.m4000066400000000000000000000113111323713302100250300ustar00rootroot00000000000000# ============================================================================ # http://www.gnu.org/software/autoconf-archive/ax_boost_program_options.html # ============================================================================ # # SYNOPSIS # # AX_BOOST_PROGRAM_OPTIONS # # DESCRIPTION # # Test for program options library from the Boost C++ libraries. The macro # requires a preceding call to AX_BOOST_BASE. Further documentation is # available at . # # This macro calls: # # AC_SUBST(BOOST_PROGRAM_OPTIONS_LIB) # # And sets: # # HAVE_BOOST_PROGRAM_OPTIONS # # LICENSE # # Copyright (c) 2009 Thomas Porschberg # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice # and this notice are preserved. This file is offered as-is, without any # warranty. #serial 20 AC_DEFUN([AX_BOOST_PROGRAM_OPTIONS], [ AC_ARG_WITH([boost-program-options], AS_HELP_STRING([--with-boost-program-options@<:@=special-lib@:>@], [use the program options library from boost - it is possible to specify a certain library for the linker e.g. --with-boost-program-options=boost_program_options-gcc-mt-1_33_1 ]), [ if test "$withval" = "no"; then want_boost="no" elif test "$withval" = "yes"; then want_boost="yes" ax_boost_user_program_options_lib="" else want_boost="yes" ax_boost_user_program_options_lib="$withval" fi ], [want_boost="yes"] ) if test "x$want_boost" = "xyes"; then AC_REQUIRE([AC_PROG_CC]) export want_boost CPPFLAGS_SAVED="$CPPFLAGS" CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS" export CPPFLAGS LDFLAGS_SAVED="$LDFLAGS" LDFLAGS="$LDFLAGS $BOOST_LDFLAGS" export LDFLAGS AC_CACHE_CHECK([whether the Boost::Program_Options library is available], ax_cv_boost_program_options, [AC_LANG_PUSH(C++) AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[@%:@include ]], [[boost::program_options::options_description generic("Generic options"); return 0;]])], ax_cv_boost_program_options=yes, ax_cv_boost_program_options=no) AC_LANG_POP([C++]) ]) if test "$ax_cv_boost_program_options" = yes; then AC_DEFINE(HAVE_BOOST_PROGRAM_OPTIONS,,[define if the Boost::PROGRAM_OPTIONS library is available]) BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/@<:@^\/@:>@*//'` if test "x$ax_boost_user_program_options_lib" = "x"; then for libextension in `ls $BOOSTLIBDIR/libboost_program_options*.so* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^lib\(boost_program_options.*\)\.so.*$;\1;'` `ls $BOOSTLIBDIR/libboost_program_options*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^lib\(boost_program_options.*\)\.a*$;\1;'` ; do ax_lib=${libextension} AC_CHECK_LIB($ax_lib, exit, [BOOST_PROGRAM_OPTIONS_LIB="-l$ax_lib"; AC_SUBST(BOOST_PROGRAM_OPTIONS_LIB) link_program_options="yes"; break], [link_program_options="no"]) done if test "x$link_program_options" != "xyes"; then for libextension in `ls $BOOSTLIBDIR/boost_program_options*.dll* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^\(boost_program_options.*\)\.dll.*$;\1;'` `ls $BOOSTLIBDIR/boost_program_options*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^\(boost_program_options.*\)\.a*$;\1;'` ; do ax_lib=${libextension} AC_CHECK_LIB($ax_lib, exit, [BOOST_PROGRAM_OPTIONS_LIB="-l$ax_lib"; AC_SUBST(BOOST_PROGRAM_OPTIONS_LIB) link_program_options="yes"; break], [link_program_options="no"]) done fi else for ax_lib in $ax_boost_user_program_options_lib boost_program_options-$ax_boost_user_program_options_lib; do AC_CHECK_LIB($ax_lib, main, [BOOST_PROGRAM_OPTIONS_LIB="-l$ax_lib"; AC_SUBST(BOOST_PROGRAM_OPTIONS_LIB) link_program_options="yes"; break], [link_program_options="no"]) done fi if test "x$ax_lib" = "x"; then AC_MSG_ERROR(Could not find a version of the library!) fi if test "x$link_program_options" != "xyes"; then AC_MSG_ERROR([Could not link against [$ax_lib] !]) fi fi CPPFLAGS="$CPPFLAGS_SAVED" LDFLAGS="$LDFLAGS_SAVED" fi ]) vowpal-wabbit-8.5.0.dfsg1/acinclude.d/ax_check_zlib.m4000066400000000000000000000120651323713302100225040ustar00rootroot00000000000000# =========================================================================== # http://www.gnu.org/software/autoconf-archive/ax_check_zlib.html # =========================================================================== # # SYNOPSIS # # AX_CHECK_ZLIB() # # DESCRIPTION # # This macro searches for an installed zlib library. If nothing was # specified when calling configure, it searches first in /usr/local and # then in /usr, /opt/local and /sw. If the --with-zlib=DIR is specified, # it will try to find it in DIR/include/zlib.h and DIR/lib/libz.a. If # --without-zlib is specified, the library is not searched at all. # # If either the header file (zlib.h) or the library (libz) is not found, # the configuration exits on error, asking for a valid zlib installation # directory or --without-zlib. # # The macro defines the symbol HAVE_LIBZ if the library is found. You # should use autoheader to include a definition for this symbol in a # config.h file. Sample usage in a C/C++ source is as follows: # # #ifdef HAVE_LIBZ # #include # #endif /* HAVE_LIBZ */ # # LICENSE # # Copyright (c) 2008 Loic Dachary # Copyright (c) 2010 Bastien Chevreux # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; either version 2 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see . # # As a special exception, the respective Autoconf Macro's copyright owner # gives unlimited permission to copy, distribute and modify the configure # scripts that are the output of Autoconf when processing the Macro. You # need not follow the terms of the GNU General Public License when using # or distributing such scripts, even though portions of the text of the # Macro appear in them. The GNU General Public License (GPL) does govern # all other use of the material that constitutes the Autoconf Macro. # # This special exception to the GPL applies to versions of the Autoconf # Macro released by the Autoconf Archive. When you make and distribute a # modified version of the Autoconf Macro, you may extend this special # exception to the GPL to apply to your modified version as well. #serial 8 AU_ALIAS([CHECK_ZLIB], [AX_CHECK_ZLIB]) AC_DEFUN([AX_CHECK_ZLIB], # # Handle user hints # [AC_MSG_CHECKING(if zlib is wanted) AC_ARG_WITH(zlib, [ --with-zlib=DIR root directory path of zlib installation [defaults to /usr/local or /usr if not found in /usr/local] --without-zlib to disable zlib usage completely], [if test "$withval" != no ; then zlib_places="/usr/local /usr /opt/local /sw" AC_MSG_RESULT(yes) if test -d "$withval" then zlib_places="$withval $zlib_places" else AC_MSG_WARN([Sorry, $withval does not exist, checking usual places]) fi else AC_MSG_RESULT(no) fi], [AC_MSG_RESULT(yes)]) # # Locate zlib, if wanted # if test -n "${zlib_places}" then # check the user supplied or any other more or less 'standard' place: # Most UNIX systems : /usr/local and /usr # MacPorts / Fink on OSX : /opt/local respectively /sw for ZLIB_HOME in ${zlib_places} ; do if test -f "${ZLIB_HOME}/include/zlib.h"; then break; fi ZLIB_HOME="" done # if zlib.h was nowhere to be found, give a notice and bail out if test ! -n "${ZLIB_HOME}"; then AC_MSG_ERROR(No zlib.h in any include directory of ${zlib_places}: either specify a valid zlib installation with --with-zlib=DIR or disable zlib usage with --without-zlib) fi ZLIB_OLD_LDFLAGS=$LDFLAGS ZLIB_OLD_CPPFLAGS=$LDFLAGS LDFLAGS="$LDFLAGS -L${ZLIB_HOME}/lib" CPPFLAGS="$CPPFLAGS -I${ZLIB_HOME}/include" AC_LANG_SAVE AC_LANG_C AC_CHECK_LIB(z, inflateEnd, [zlib_cv_libz=yes], [zlib_cv_libz=no]) AC_CHECK_HEADER(zlib.h, [zlib_cv_zlib_h=yes], [zlib_cv_zlib_h=no]) AC_LANG_RESTORE if test "$zlib_cv_libz" = "yes" -a "$zlib_cv_zlib_h" = "yes" then # # If both library and header were found, use them # AC_CHECK_LIB(z, inflateEnd) AC_MSG_CHECKING(zlib in ${ZLIB_HOME}) AC_MSG_RESULT(ok) else # # If either header or library was not found, revert and bomb # AC_MSG_CHECKING(zlib in ${ZLIB_HOME}) LDFLAGS="$ZLIB_OLD_LDFLAGS" CPPFLAGS="$ZLIB_OLD_CPPFLAGS" AC_MSG_RESULT(failed) AC_MSG_ERROR(either specify a valid zlib installation with --with-zlib=DIR or disable zlib usage with --without-zlib) fi fi ]) vowpal-wabbit-8.5.0.dfsg1/acinclude.d/ax_cxx_compile_stdcxx_11.m4000066400000000000000000000112751323713302100246210ustar00rootroot00000000000000# ============================================================================ # http://www.gnu.org/software/autoconf-archive/ax_cxx_compile_stdcxx_11.html # ============================================================================ # # SYNOPSIS # # AX_CXX_COMPILE_STDCXX_11([ext|noext],[mandatory|optional]) # # DESCRIPTION # # Check for baseline language coverage in the compiler for the C++11 # standard; if necessary, add switches to CXXFLAGS to enable support. # # The first argument, if specified, indicates whether you insist on an # extended mode (e.g. -std=gnu++11) or a strict conformance mode (e.g. # -std=c++11). If neither is specified, you get whatever works, with # preference for an extended mode. # # The second argument, if specified 'mandatory' or if left unspecified, # indicates that baseline C++11 support is required and that the macro # should error out if no mode with that support is found. If specified # 'optional', then configuration proceeds regardless, after defining # HAVE_CXX11 if and only if a supporting mode is found. # # LICENSE # # Copyright (c) 2008 Benjamin Kosnik # Copyright (c) 2012 Zack Weinberg # Copyright (c) 2013 Roy Stogner # Copyright (c) 2014 Alexey Sokolov # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice # and this notice are preserved. This file is offered as-is, without any # warranty. #serial 4 m4_define([_AX_CXX_COMPILE_STDCXX_11_testbody], [[ template struct check { static_assert(sizeof(int) <= sizeof(T), "not big enough"); }; struct Base { virtual void f() {} }; struct Child : public Base { virtual void f() override {} }; typedef check> right_angle_brackets; int a; decltype(a) b; typedef check check_type; check_type c; check_type&& cr = static_cast(c); auto d = a; auto l = [](){}; ]]) AC_DEFUN([AX_CXX_COMPILE_STDCXX_11], [dnl m4_if([$1], [], [], [$1], [ext], [], [$1], [noext], [], [m4_fatal([invalid argument `$1' to AX_CXX_COMPILE_STDCXX_11])])dnl m4_if([$2], [], [ax_cxx_compile_cxx11_required=true], [$2], [mandatory], [ax_cxx_compile_cxx11_required=true], [$2], [optional], [ax_cxx_compile_cxx11_required=false], [m4_fatal([invalid second argument `$2' to AX_CXX_COMPILE_STDCXX_11])]) AC_LANG_PUSH([C++])dnl ac_success=no AC_CACHE_CHECK(whether $CXX supports C++11 features by default, ax_cv_cxx_compile_cxx11, [AC_COMPILE_IFELSE([AC_LANG_SOURCE([_AX_CXX_COMPILE_STDCXX_11_testbody])], [ax_cv_cxx_compile_cxx11=yes], [ax_cv_cxx_compile_cxx11=no])]) if test x$ax_cv_cxx_compile_cxx11 = xyes; then ac_success=yes fi m4_if([$1], [noext], [], [dnl if test x$ac_success = xno; then for switch in -std=gnu++11 -std=gnu++0x; do cachevar=AS_TR_SH([ax_cv_cxx_compile_cxx11_$switch]) AC_CACHE_CHECK(whether $CXX supports C++11 features with $switch, $cachevar, [ac_save_CXXFLAGS="$CXXFLAGS" CXXFLAGS="$CXXFLAGS $switch" AC_COMPILE_IFELSE([AC_LANG_SOURCE([_AX_CXX_COMPILE_STDCXX_11_testbody])], [eval $cachevar=yes], [eval $cachevar=no]) CXXFLAGS="$ac_save_CXXFLAGS"]) if eval test x\$$cachevar = xyes; then CXXFLAGS="$CXXFLAGS $switch" ac_success=yes break fi done fi]) m4_if([$1], [ext], [], [dnl if test x$ac_success = xno; then for switch in -std=c++11 -std=c++0x; do cachevar=AS_TR_SH([ax_cv_cxx_compile_cxx11_$switch]) AC_CACHE_CHECK(whether $CXX supports C++11 features with $switch, $cachevar, [ac_save_CXXFLAGS="$CXXFLAGS" CXXFLAGS="$CXXFLAGS $switch" AC_COMPILE_IFELSE([AC_LANG_SOURCE([_AX_CXX_COMPILE_STDCXX_11_testbody])], [eval $cachevar=yes], [eval $cachevar=no]) CXXFLAGS="$ac_save_CXXFLAGS"]) if eval test x\$$cachevar = xyes; then CXXFLAGS="$CXXFLAGS $switch" ac_success=yes break fi done fi]) AC_LANG_POP([C++]) if test x$ax_cxx_compile_cxx11_required = xtrue; then if test x$ac_success = xno; then AC_MSG_ERROR([*** A compiler with support for C++11 language features is required.]) fi else if test x$ac_success = xno; then HAVE_CXX11=0 AC_MSG_NOTICE([No compiler with C++11 support was found]) else HAVE_CXX11=1 AC_DEFINE(HAVE_CXX11,1, [define if the compiler supports basic C++11 syntax]) fi AC_SUBST(HAVE_CXX11) fi ]) vowpal-wabbit-8.5.0.dfsg1/acinclude.d/ax_pthread.m4000066400000000000000000000304401323713302100220330ustar00rootroot00000000000000# =========================================================================== # http://www.gnu.org/software/autoconf-archive/ax_pthread.html # =========================================================================== # # SYNOPSIS # # AX_PTHREAD([ACTION-IF-FOUND[, ACTION-IF-NOT-FOUND]]) # # DESCRIPTION # # This macro figures out how to build C programs using POSIX threads. It # sets the PTHREAD_LIBS output variable to the threads library and linker # flags, and the PTHREAD_CFLAGS output variable to any special C compiler # flags that are needed. (The user can also force certain compiler # flags/libs to be tested by setting these environment variables.) # # Also sets PTHREAD_CC to any special C compiler that is needed for # multi-threaded programs (defaults to the value of CC otherwise). (This # is necessary on AIX to use the special cc_r compiler alias.) # # NOTE: You are assumed to not only compile your program with these flags, # but also link it with them as well. e.g. you should link with # $PTHREAD_CC $CFLAGS $PTHREAD_CFLAGS $LDFLAGS ... $PTHREAD_LIBS $LIBS # # If you are only building threads programs, you may wish to use these # variables in your default LIBS, CFLAGS, and CC: # # LIBS="$PTHREAD_LIBS $LIBS" # CFLAGS="$CFLAGS $PTHREAD_CFLAGS" # CC="$PTHREAD_CC" # # In addition, if the PTHREAD_CREATE_JOINABLE thread-attribute constant # has a nonstandard name, defines PTHREAD_CREATE_JOINABLE to that name # (e.g. PTHREAD_CREATE_UNDETACHED on AIX). # # Also HAVE_PTHREAD_PRIO_INHERIT is defined if pthread is found and the # PTHREAD_PRIO_INHERIT symbol is defined when compiling with # PTHREAD_CFLAGS. # # ACTION-IF-FOUND is a list of shell commands to run if a threads library # is found, and ACTION-IF-NOT-FOUND is a list of commands to run it if it # is not found. If ACTION-IF-FOUND is not specified, the default action # will define HAVE_PTHREAD. # # Please let the authors know if this macro fails on any platform, or if # you have any other suggestions or comments. This macro was based on work # by SGJ on autoconf scripts for FFTW (http://www.fftw.org/) (with help # from M. Frigo), as well as ac_pthread and hb_pthread macros posted by # Alejandro Forero Cuervo to the autoconf macro repository. We are also # grateful for the helpful feedback of numerous users. # # Updated for Autoconf 2.68 by Daniel Richard G. # # LICENSE # # Copyright (c) 2008 Steven G. Johnson # Copyright (c) 2011 Daniel Richard G. # # This program is free software: you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation, either version 3 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see . # # As a special exception, the respective Autoconf Macro's copyright owner # gives unlimited permission to copy, distribute and modify the configure # scripts that are the output of Autoconf when processing the Macro. You # need not follow the terms of the GNU General Public License when using # or distributing such scripts, even though portions of the text of the # Macro appear in them. The GNU General Public License (GPL) does govern # all other use of the material that constitutes the Autoconf Macro. # # This special exception to the GPL applies to versions of the Autoconf # Macro released by the Autoconf Archive. When you make and distribute a # modified version of the Autoconf Macro, you may extend this special # exception to the GPL to apply to your modified version as well. #serial 17 AU_ALIAS([ACX_PTHREAD], [AX_PTHREAD]) AC_DEFUN([AX_PTHREAD], [ AC_REQUIRE([AC_CANONICAL_HOST]) AC_LANG_PUSH([C]) ax_pthread_ok=no # We used to check for pthread.h first, but this fails if pthread.h # requires special compiler flags (e.g. on True64 or Sequent). # It gets checked for in the link test anyway. # First of all, check if the user has set any of the PTHREAD_LIBS, # etcetera environment variables, and if threads linking works using # them: if test x"$PTHREAD_LIBS$PTHREAD_CFLAGS" != x; then save_CFLAGS="$CFLAGS" CFLAGS="$CFLAGS $PTHREAD_CFLAGS" save_LIBS="$LIBS" LIBS="$PTHREAD_LIBS $LIBS" AC_MSG_CHECKING([for pthread_join in LIBS=$PTHREAD_LIBS with CFLAGS=$PTHREAD_CFLAGS]) AC_TRY_LINK_FUNC(pthread_join, ax_pthread_ok=yes) AC_MSG_RESULT($ax_pthread_ok) if test x"$ax_pthread_ok" = xno; then PTHREAD_LIBS="" PTHREAD_CFLAGS="" fi LIBS="$save_LIBS" CFLAGS="$save_CFLAGS" fi # We must check for the threads library under a number of different # names; the ordering is very important because some systems # (e.g. DEC) have both -lpthread and -lpthreads, where one of the # libraries is broken (non-POSIX). # Create a list of thread flags to try. Items starting with a "-" are # C compiler flags, and other items are library names, except for "none" # which indicates that we try without any flags at all, and "pthread-config" # which is a program returning the flags for the Pth emulation library. ax_pthread_flags="pthreads none -Kthread -kthread lthread -pthread -pthreads -mthreads pthread --thread-safe -mt pthread-config" # The ordering *is* (sometimes) important. Some notes on the # individual items follow: # pthreads: AIX (must check this before -lpthread) # none: in case threads are in libc; should be tried before -Kthread and # other compiler flags to prevent continual compiler warnings # -Kthread: Sequent (threads in libc, but -Kthread needed for pthread.h) # -kthread: FreeBSD kernel threads (preferred to -pthread since SMP-able) # lthread: LinuxThreads port on FreeBSD (also preferred to -pthread) # -pthread: Linux/gcc (kernel threads), BSD/gcc (userland threads) # -pthreads: Solaris/gcc # -mthreads: Mingw32/gcc, Lynx/gcc # -mt: Sun Workshop C (may only link SunOS threads [-lthread], but it # doesn't hurt to check since this sometimes defines pthreads too; # also defines -D_REENTRANT) # ... -mt is also the pthreads flag for HP/aCC # pthread: Linux, etcetera # --thread-safe: KAI C++ # pthread-config: use pthread-config program (for GNU Pth library) case "${host_cpu}-${host_os}" in *solaris*) # On Solaris (at least, for some versions), libc contains stubbed # (non-functional) versions of the pthreads routines, so link-based # tests will erroneously succeed. (We need to link with -pthreads/-mt/ # -lpthread.) (The stubs are missing pthread_cleanup_push, or rather # a function called by this macro, so we could check for that, but # who knows whether they'll stub that too in a future libc.) So, # we'll just look for -pthreads and -lpthread first: ax_pthread_flags="-pthreads pthread -mt -pthread $ax_pthread_flags" ;; *-darwin*) ax_pthread_flags="-pthread $ax_pthread_flags" ;; esac if test x"$ax_pthread_ok" = xno; then for flag in $ax_pthread_flags; do case $flag in none) AC_MSG_CHECKING([whether pthreads work without any flags]) ;; -*) AC_MSG_CHECKING([whether pthreads work with $flag]) PTHREAD_CFLAGS="$flag" ;; pthread-config) AC_CHECK_PROG(ax_pthread_config, pthread-config, yes, no) if test x"$ax_pthread_config" = xno; then continue; fi PTHREAD_CFLAGS="`pthread-config --cflags`" PTHREAD_LIBS="`pthread-config --ldflags` `pthread-config --libs`" ;; *) AC_MSG_CHECKING([for the pthreads library -l$flag]) PTHREAD_LIBS="-l$flag" ;; esac save_LIBS="$LIBS" save_CFLAGS="$CFLAGS" LIBS="$PTHREAD_LIBS $LIBS" CFLAGS="$CFLAGS $PTHREAD_CFLAGS" # Check for various functions. We must include pthread.h, # since some functions may be macros. (On the Sequent, we # need a special flag -Kthread to make this header compile.) # We check for pthread_join because it is in -lpthread on IRIX # while pthread_create is in libc. We check for pthread_attr_init # due to DEC craziness with -lpthreads. We check for # pthread_cleanup_push because it is one of the few pthread # functions on Solaris that doesn't have a non-functional libc stub. # We try pthread_create on general principles. AC_LINK_IFELSE([AC_LANG_PROGRAM([#include static void routine(void *a) { a = 0; } static void *start_routine(void *a) { return a; }], [pthread_t th; pthread_attr_t attr; pthread_create(&th, 0, start_routine, 0); pthread_join(th, 0); pthread_attr_init(&attr); pthread_cleanup_push(routine, 0); pthread_cleanup_pop(0) /* ; */])], [ax_pthread_ok=yes], []) LIBS="$save_LIBS" CFLAGS="$save_CFLAGS" AC_MSG_RESULT($ax_pthread_ok) if test "x$ax_pthread_ok" = xyes; then break; fi PTHREAD_LIBS="" PTHREAD_CFLAGS="" done fi # Various other checks: if test "x$ax_pthread_ok" = xyes; then save_LIBS="$LIBS" LIBS="$PTHREAD_LIBS $LIBS" save_CFLAGS="$CFLAGS" CFLAGS="$CFLAGS $PTHREAD_CFLAGS" # Detect AIX lossage: JOINABLE attribute is called UNDETACHED. AC_MSG_CHECKING([for joinable pthread attribute]) attr_name=unknown for attr in PTHREAD_CREATE_JOINABLE PTHREAD_CREATE_UNDETACHED; do AC_LINK_IFELSE([AC_LANG_PROGRAM([#include ], [int attr = $attr; return attr /* ; */])], [attr_name=$attr; break], []) done AC_MSG_RESULT($attr_name) if test "$attr_name" != PTHREAD_CREATE_JOINABLE; then AC_DEFINE_UNQUOTED(PTHREAD_CREATE_JOINABLE, $attr_name, [Define to necessary symbol if this constant uses a non-standard name on your system.]) fi AC_MSG_CHECKING([if more special flags are required for pthreads]) flag=no case "${host_cpu}-${host_os}" in *-aix* | *-freebsd* | *-darwin*) flag="-D_THREAD_SAFE";; *-osf* | *-hpux*) flag="-D_REENTRANT";; *solaris*) if test "$GCC" = "yes"; then flag="-D_REENTRANT" else flag="-mt -D_REENTRANT" fi ;; esac AC_MSG_RESULT(${flag}) if test "x$flag" != xno; then PTHREAD_CFLAGS="$flag $PTHREAD_CFLAGS" fi AC_CACHE_CHECK([for PTHREAD_PRIO_INHERIT], ax_cv_PTHREAD_PRIO_INHERIT, [ AC_LINK_IFELSE([ AC_LANG_PROGRAM([[#include ]], [[int i = PTHREAD_PRIO_INHERIT;]])], [ax_cv_PTHREAD_PRIO_INHERIT=yes], [ax_cv_PTHREAD_PRIO_INHERIT=no]) ]) AS_IF([test "x$ax_cv_PTHREAD_PRIO_INHERIT" = "xyes"], AC_DEFINE([HAVE_PTHREAD_PRIO_INHERIT], 1, [Have PTHREAD_PRIO_INHERIT.])) LIBS="$save_LIBS" CFLAGS="$save_CFLAGS" # More AIX lossage: must compile with xlc_r or cc_r if test x"$GCC" != xyes; then AC_CHECK_PROGS(PTHREAD_CC, xlc_r cc_r, ${CC}) else PTHREAD_CC=$CC fi else PTHREAD_CC="$CC" fi AC_SUBST(PTHREAD_LIBS) AC_SUBST(PTHREAD_CFLAGS) AC_SUBST(PTHREAD_CC) # Finally, execute ACTION-IF-FOUND/ACTION-IF-NOT-FOUND: if test x"$ax_pthread_ok" = xyes; then ifelse([$1],,AC_DEFINE(HAVE_PTHREAD,1,[Define if you have POSIX threads libraries and header files.]),[$1]) : else ax_pthread_ok=no $2 fi AC_LANG_POP ])dnl AX_PTHREAD vowpal-wabbit-8.5.0.dfsg1/acinclude.d/libtool.m4000066400000000000000000011263111323713302100213640ustar00rootroot00000000000000# libtool.m4 - Configure libtool for the host system. -*-Autoconf-*- # # Copyright (C) 1996-2001, 2003-2015 Free Software Foundation, Inc. # Written by Gordon Matzigkeit, 1996 # # This file is free software; the Free Software Foundation gives # unlimited permission to copy and/or distribute it, with or without # modifications, as long as this notice is preserved. m4_define([_LT_COPYING], [dnl # Copyright (C) 2014 Free Software Foundation, Inc. # This is free software; see the source for copying conditions. There is NO # warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # GNU Libtool is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of of the License, or # (at your option) any later version. # # As a special exception to the GNU General Public License, if you # distribute this file as part of a program or library that is built # using GNU Libtool, you may include this file under the same # distribution terms that you use for the rest of that program. # # GNU Libtool is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . ]) # serial 58 LT_INIT # LT_PREREQ(VERSION) # ------------------ # Complain and exit if this libtool version is less that VERSION. m4_defun([LT_PREREQ], [m4_if(m4_version_compare(m4_defn([LT_PACKAGE_VERSION]), [$1]), -1, [m4_default([$3], [m4_fatal([Libtool version $1 or higher is required], 63)])], [$2])]) # _LT_CHECK_BUILDDIR # ------------------ # Complain if the absolute build directory name contains unusual characters m4_defun([_LT_CHECK_BUILDDIR], [case `pwd` in *\ * | *\ *) AC_MSG_WARN([Libtool does not cope well with whitespace in `pwd`]) ;; esac ]) # LT_INIT([OPTIONS]) # ------------------ AC_DEFUN([LT_INIT], [AC_PREREQ([2.62])dnl We use AC_PATH_PROGS_FEATURE_CHECK AC_REQUIRE([AC_CONFIG_AUX_DIR_DEFAULT])dnl AC_BEFORE([$0], [LT_LANG])dnl AC_BEFORE([$0], [LT_OUTPUT])dnl AC_BEFORE([$0], [LTDL_INIT])dnl m4_require([_LT_CHECK_BUILDDIR])dnl dnl Autoconf doesn't catch unexpanded LT_ macros by default: m4_pattern_forbid([^_?LT_[A-Z_]+$])dnl m4_pattern_allow([^(_LT_EOF|LT_DLGLOBAL|LT_DLLAZY_OR_NOW|LT_MULTI_MODULE)$])dnl dnl aclocal doesn't pull ltoptions.m4, ltsugar.m4, or ltversion.m4 dnl unless we require an AC_DEFUNed macro: AC_REQUIRE([LTOPTIONS_VERSION])dnl AC_REQUIRE([LTSUGAR_VERSION])dnl AC_REQUIRE([LTVERSION_VERSION])dnl AC_REQUIRE([LTOBSOLETE_VERSION])dnl m4_require([_LT_PROG_LTMAIN])dnl _LT_SHELL_INIT([SHELL=${CONFIG_SHELL-/bin/sh}]) dnl Parse OPTIONS _LT_SET_OPTIONS([$0], [$1]) # This can be used to rebuild libtool when needed LIBTOOL_DEPS=$ltmain # Always use our own libtool. LIBTOOL='$(SHELL) $(top_builddir)/libtool' AC_SUBST(LIBTOOL)dnl _LT_SETUP # Only expand once: m4_define([LT_INIT]) ])# LT_INIT # Old names: AU_ALIAS([AC_PROG_LIBTOOL], [LT_INIT]) AU_ALIAS([AM_PROG_LIBTOOL], [LT_INIT]) dnl aclocal-1.4 backwards compatibility: dnl AC_DEFUN([AC_PROG_LIBTOOL], []) dnl AC_DEFUN([AM_PROG_LIBTOOL], []) # _LT_PREPARE_CC_BASENAME # ----------------------- m4_defun([_LT_PREPARE_CC_BASENAME], [ # Calculate cc_basename. Skip known compiler wrappers and cross-prefix. func_cc_basename () { for cc_temp in @S|@*""; do case $cc_temp in compile | *[[\\/]]compile | ccache | *[[\\/]]ccache ) ;; distcc | *[[\\/]]distcc | purify | *[[\\/]]purify ) ;; \-*) ;; *) break;; esac done func_cc_basename_result=`$ECHO "$cc_temp" | $SED "s%.*/%%; s%^$host_alias-%%"` } ])# _LT_PREPARE_CC_BASENAME # _LT_CC_BASENAME(CC) # ------------------- # It would be clearer to call AC_REQUIREs from _LT_PREPARE_CC_BASENAME, # but that macro is also expanded into generated libtool script, which # arranges for $SED and $ECHO to be set by different means. m4_defun([_LT_CC_BASENAME], [m4_require([_LT_PREPARE_CC_BASENAME])dnl AC_REQUIRE([_LT_DECL_SED])dnl AC_REQUIRE([_LT_PROG_ECHO_BACKSLASH])dnl func_cc_basename $1 cc_basename=$func_cc_basename_result ]) # _LT_FILEUTILS_DEFAULTS # ---------------------- # It is okay to use these file commands and assume they have been set # sensibly after 'm4_require([_LT_FILEUTILS_DEFAULTS])'. m4_defun([_LT_FILEUTILS_DEFAULTS], [: ${CP="cp -f"} : ${MV="mv -f"} : ${RM="rm -f"} ])# _LT_FILEUTILS_DEFAULTS # _LT_SETUP # --------- m4_defun([_LT_SETUP], [AC_REQUIRE([AC_CANONICAL_HOST])dnl AC_REQUIRE([AC_CANONICAL_BUILD])dnl AC_REQUIRE([_LT_PREPARE_SED_QUOTE_VARS])dnl AC_REQUIRE([_LT_PROG_ECHO_BACKSLASH])dnl _LT_DECL([], [PATH_SEPARATOR], [1], [The PATH separator for the build system])dnl dnl _LT_DECL([], [host_alias], [0], [The host system])dnl _LT_DECL([], [host], [0])dnl _LT_DECL([], [host_os], [0])dnl dnl _LT_DECL([], [build_alias], [0], [The build system])dnl _LT_DECL([], [build], [0])dnl _LT_DECL([], [build_os], [0])dnl dnl AC_REQUIRE([AC_PROG_CC])dnl AC_REQUIRE([LT_PATH_LD])dnl AC_REQUIRE([LT_PATH_NM])dnl dnl AC_REQUIRE([AC_PROG_LN_S])dnl test -z "$LN_S" && LN_S="ln -s" _LT_DECL([], [LN_S], [1], [Whether we need soft or hard links])dnl dnl AC_REQUIRE([LT_CMD_MAX_LEN])dnl _LT_DECL([objext], [ac_objext], [0], [Object file suffix (normally "o")])dnl _LT_DECL([], [exeext], [0], [Executable file suffix (normally "")])dnl dnl m4_require([_LT_FILEUTILS_DEFAULTS])dnl m4_require([_LT_CHECK_SHELL_FEATURES])dnl m4_require([_LT_PATH_CONVERSION_FUNCTIONS])dnl m4_require([_LT_CMD_RELOAD])dnl m4_require([_LT_CHECK_MAGIC_METHOD])dnl m4_require([_LT_CHECK_SHAREDLIB_FROM_LINKLIB])dnl m4_require([_LT_CMD_OLD_ARCHIVE])dnl m4_require([_LT_CMD_GLOBAL_SYMBOLS])dnl m4_require([_LT_WITH_SYSROOT])dnl m4_require([_LT_CMD_TRUNCATE])dnl _LT_CONFIG_LIBTOOL_INIT([ # See if we are running on zsh, and set the options that allow our # commands through without removal of \ escapes INIT. if test -n "\${ZSH_VERSION+set}"; then setopt NO_GLOB_SUBST fi ]) if test -n "${ZSH_VERSION+set}"; then setopt NO_GLOB_SUBST fi _LT_CHECK_OBJDIR m4_require([_LT_TAG_COMPILER])dnl case $host_os in aix3*) # AIX sometimes has problems with the GCC collect2 program. For some # reason, if we set the COLLECT_NAMES environment variable, the problems # vanish in a puff of smoke. if test set != "${COLLECT_NAMES+set}"; then COLLECT_NAMES= export COLLECT_NAMES fi ;; esac # Global variables: ofile=libtool can_build_shared=yes # All known linkers require a '.a' archive for static linking (except MSVC, # which needs '.lib'). libext=a with_gnu_ld=$lt_cv_prog_gnu_ld old_CC=$CC old_CFLAGS=$CFLAGS # Set sane defaults for various variables test -z "$CC" && CC=cc test -z "$LTCC" && LTCC=$CC test -z "$LTCFLAGS" && LTCFLAGS=$CFLAGS test -z "$LD" && LD=ld test -z "$ac_objext" && ac_objext=o _LT_CC_BASENAME([$compiler]) # Only perform the check for file, if the check method requires it test -z "$MAGIC_CMD" && MAGIC_CMD=file case $deplibs_check_method in file_magic*) if test "$file_magic_cmd" = '$MAGIC_CMD'; then _LT_PATH_MAGIC fi ;; esac # Use C for the default configuration in the libtool script LT_SUPPORTED_TAG([CC]) _LT_LANG_C_CONFIG _LT_LANG_DEFAULT_CONFIG _LT_CONFIG_COMMANDS ])# _LT_SETUP # _LT_PREPARE_SED_QUOTE_VARS # -------------------------- # Define a few sed substitution that help us do robust quoting. m4_defun([_LT_PREPARE_SED_QUOTE_VARS], [# Backslashify metacharacters that are still active within # double-quoted strings. sed_quote_subst='s/\([["`$\\]]\)/\\\1/g' # Same as above, but do not quote variable references. double_quote_subst='s/\([["`\\]]\)/\\\1/g' # Sed substitution to delay expansion of an escaped shell variable in a # double_quote_subst'ed string. delay_variable_subst='s/\\\\\\\\\\\$/\\\\\\$/g' # Sed substitution to delay expansion of an escaped single quote. delay_single_quote_subst='s/'\''/'\'\\\\\\\'\''/g' # Sed substitution to avoid accidental globbing in evaled expressions no_glob_subst='s/\*/\\\*/g' ]) # _LT_PROG_LTMAIN # --------------- # Note that this code is called both from 'configure', and 'config.status' # now that we use AC_CONFIG_COMMANDS to generate libtool. Notably, # 'config.status' has no value for ac_aux_dir unless we are using Automake, # so we pass a copy along to make sure it has a sensible value anyway. m4_defun([_LT_PROG_LTMAIN], [m4_ifdef([AC_REQUIRE_AUX_FILE], [AC_REQUIRE_AUX_FILE([ltmain.sh])])dnl _LT_CONFIG_LIBTOOL_INIT([ac_aux_dir='$ac_aux_dir']) ltmain=$ac_aux_dir/ltmain.sh ])# _LT_PROG_LTMAIN ## ------------------------------------- ## ## Accumulate code for creating libtool. ## ## ------------------------------------- ## # So that we can recreate a full libtool script including additional # tags, we accumulate the chunks of code to send to AC_CONFIG_COMMANDS # in macros and then make a single call at the end using the 'libtool' # label. # _LT_CONFIG_LIBTOOL_INIT([INIT-COMMANDS]) # ---------------------------------------- # Register INIT-COMMANDS to be passed to AC_CONFIG_COMMANDS later. m4_define([_LT_CONFIG_LIBTOOL_INIT], [m4_ifval([$1], [m4_append([_LT_OUTPUT_LIBTOOL_INIT], [$1 ])])]) # Initialize. m4_define([_LT_OUTPUT_LIBTOOL_INIT]) # _LT_CONFIG_LIBTOOL([COMMANDS]) # ------------------------------ # Register COMMANDS to be passed to AC_CONFIG_COMMANDS later. m4_define([_LT_CONFIG_LIBTOOL], [m4_ifval([$1], [m4_append([_LT_OUTPUT_LIBTOOL_COMMANDS], [$1 ])])]) # Initialize. m4_define([_LT_OUTPUT_LIBTOOL_COMMANDS]) # _LT_CONFIG_SAVE_COMMANDS([COMMANDS], [INIT_COMMANDS]) # ----------------------------------------------------- m4_defun([_LT_CONFIG_SAVE_COMMANDS], [_LT_CONFIG_LIBTOOL([$1]) _LT_CONFIG_LIBTOOL_INIT([$2]) ]) # _LT_FORMAT_COMMENT([COMMENT]) # ----------------------------- # Add leading comment marks to the start of each line, and a trailing # full-stop to the whole comment if one is not present already. m4_define([_LT_FORMAT_COMMENT], [m4_ifval([$1], [ m4_bpatsubst([m4_bpatsubst([$1], [^ *], [# ])], [['`$\]], [\\\&])]m4_bmatch([$1], [[!?.]$], [], [.]) )]) ## ------------------------ ## ## FIXME: Eliminate VARNAME ## ## ------------------------ ## # _LT_DECL([CONFIGNAME], VARNAME, VALUE, [DESCRIPTION], [IS-TAGGED?]) # ------------------------------------------------------------------- # CONFIGNAME is the name given to the value in the libtool script. # VARNAME is the (base) name used in the configure script. # VALUE may be 0, 1 or 2 for a computed quote escaped value based on # VARNAME. Any other value will be used directly. m4_define([_LT_DECL], [lt_if_append_uniq([lt_decl_varnames], [$2], [, ], [lt_dict_add_subkey([lt_decl_dict], [$2], [libtool_name], [m4_ifval([$1], [$1], [$2])]) lt_dict_add_subkey([lt_decl_dict], [$2], [value], [$3]) m4_ifval([$4], [lt_dict_add_subkey([lt_decl_dict], [$2], [description], [$4])]) lt_dict_add_subkey([lt_decl_dict], [$2], [tagged?], [m4_ifval([$5], [yes], [no])])]) ]) # _LT_TAGDECL([CONFIGNAME], VARNAME, VALUE, [DESCRIPTION]) # -------------------------------------------------------- m4_define([_LT_TAGDECL], [_LT_DECL([$1], [$2], [$3], [$4], [yes])]) # lt_decl_tag_varnames([SEPARATOR], [VARNAME1...]) # ------------------------------------------------ m4_define([lt_decl_tag_varnames], [_lt_decl_filter([tagged?], [yes], $@)]) # _lt_decl_filter(SUBKEY, VALUE, [SEPARATOR], [VARNAME1..]) # --------------------------------------------------------- m4_define([_lt_decl_filter], [m4_case([$#], [0], [m4_fatal([$0: too few arguments: $#])], [1], [m4_fatal([$0: too few arguments: $#: $1])], [2], [lt_dict_filter([lt_decl_dict], [$1], [$2], [], lt_decl_varnames)], [3], [lt_dict_filter([lt_decl_dict], [$1], [$2], [$3], lt_decl_varnames)], [lt_dict_filter([lt_decl_dict], $@)])[]dnl ]) # lt_decl_quote_varnames([SEPARATOR], [VARNAME1...]) # -------------------------------------------------- m4_define([lt_decl_quote_varnames], [_lt_decl_filter([value], [1], $@)]) # lt_decl_dquote_varnames([SEPARATOR], [VARNAME1...]) # --------------------------------------------------- m4_define([lt_decl_dquote_varnames], [_lt_decl_filter([value], [2], $@)]) # lt_decl_varnames_tagged([SEPARATOR], [VARNAME1...]) # --------------------------------------------------- m4_define([lt_decl_varnames_tagged], [m4_assert([$# <= 2])dnl _$0(m4_quote(m4_default([$1], [[, ]])), m4_ifval([$2], [[$2]], [m4_dquote(lt_decl_tag_varnames)]), m4_split(m4_normalize(m4_quote(_LT_TAGS)), [ ]))]) m4_define([_lt_decl_varnames_tagged], [m4_ifval([$3], [lt_combine([$1], [$2], [_], $3)])]) # lt_decl_all_varnames([SEPARATOR], [VARNAME1...]) # ------------------------------------------------ m4_define([lt_decl_all_varnames], [_$0(m4_quote(m4_default([$1], [[, ]])), m4_if([$2], [], m4_quote(lt_decl_varnames), m4_quote(m4_shift($@))))[]dnl ]) m4_define([_lt_decl_all_varnames], [lt_join($@, lt_decl_varnames_tagged([$1], lt_decl_tag_varnames([[, ]], m4_shift($@))))dnl ]) # _LT_CONFIG_STATUS_DECLARE([VARNAME]) # ------------------------------------ # Quote a variable value, and forward it to 'config.status' so that its # declaration there will have the same value as in 'configure'. VARNAME # must have a single quote delimited value for this to work. m4_define([_LT_CONFIG_STATUS_DECLARE], [$1='`$ECHO "$][$1" | $SED "$delay_single_quote_subst"`']) # _LT_CONFIG_STATUS_DECLARATIONS # ------------------------------ # We delimit libtool config variables with single quotes, so when # we write them to config.status, we have to be sure to quote all # embedded single quotes properly. In configure, this macro expands # each variable declared with _LT_DECL (and _LT_TAGDECL) into: # # ='`$ECHO "$" | $SED "$delay_single_quote_subst"`' m4_defun([_LT_CONFIG_STATUS_DECLARATIONS], [m4_foreach([_lt_var], m4_quote(lt_decl_all_varnames), [m4_n([_LT_CONFIG_STATUS_DECLARE(_lt_var)])])]) # _LT_LIBTOOL_TAGS # ---------------- # Output comment and list of tags supported by the script m4_defun([_LT_LIBTOOL_TAGS], [_LT_FORMAT_COMMENT([The names of the tagged configurations supported by this script])dnl available_tags='_LT_TAGS'dnl ]) # _LT_LIBTOOL_DECLARE(VARNAME, [TAG]) # ----------------------------------- # Extract the dictionary values for VARNAME (optionally with TAG) and # expand to a commented shell variable setting: # # # Some comment about what VAR is for. # visible_name=$lt_internal_name m4_define([_LT_LIBTOOL_DECLARE], [_LT_FORMAT_COMMENT(m4_quote(lt_dict_fetch([lt_decl_dict], [$1], [description])))[]dnl m4_pushdef([_libtool_name], m4_quote(lt_dict_fetch([lt_decl_dict], [$1], [libtool_name])))[]dnl m4_case(m4_quote(lt_dict_fetch([lt_decl_dict], [$1], [value])), [0], [_libtool_name=[$]$1], [1], [_libtool_name=$lt_[]$1], [2], [_libtool_name=$lt_[]$1], [_libtool_name=lt_dict_fetch([lt_decl_dict], [$1], [value])])[]dnl m4_ifval([$2], [_$2])[]m4_popdef([_libtool_name])[]dnl ]) # _LT_LIBTOOL_CONFIG_VARS # ----------------------- # Produce commented declarations of non-tagged libtool config variables # suitable for insertion in the LIBTOOL CONFIG section of the 'libtool' # script. Tagged libtool config variables (even for the LIBTOOL CONFIG # section) are produced by _LT_LIBTOOL_TAG_VARS. m4_defun([_LT_LIBTOOL_CONFIG_VARS], [m4_foreach([_lt_var], m4_quote(_lt_decl_filter([tagged?], [no], [], lt_decl_varnames)), [m4_n([_LT_LIBTOOL_DECLARE(_lt_var)])])]) # _LT_LIBTOOL_TAG_VARS(TAG) # ------------------------- m4_define([_LT_LIBTOOL_TAG_VARS], [m4_foreach([_lt_var], m4_quote(lt_decl_tag_varnames), [m4_n([_LT_LIBTOOL_DECLARE(_lt_var, [$1])])])]) # _LT_TAGVAR(VARNAME, [TAGNAME]) # ------------------------------ m4_define([_LT_TAGVAR], [m4_ifval([$2], [$1_$2], [$1])]) # _LT_CONFIG_COMMANDS # ------------------- # Send accumulated output to $CONFIG_STATUS. Thanks to the lists of # variables for single and double quote escaping we saved from calls # to _LT_DECL, we can put quote escaped variables declarations # into 'config.status', and then the shell code to quote escape them in # for loops in 'config.status'. Finally, any additional code accumulated # from calls to _LT_CONFIG_LIBTOOL_INIT is expanded. m4_defun([_LT_CONFIG_COMMANDS], [AC_PROVIDE_IFELSE([LT_OUTPUT], dnl If the libtool generation code has been placed in $CONFIG_LT, dnl instead of duplicating it all over again into config.status, dnl then we will have config.status run $CONFIG_LT later, so it dnl needs to know what name is stored there: [AC_CONFIG_COMMANDS([libtool], [$SHELL $CONFIG_LT || AS_EXIT(1)], [CONFIG_LT='$CONFIG_LT'])], dnl If the libtool generation code is destined for config.status, dnl expand the accumulated commands and init code now: [AC_CONFIG_COMMANDS([libtool], [_LT_OUTPUT_LIBTOOL_COMMANDS], [_LT_OUTPUT_LIBTOOL_COMMANDS_INIT])]) ])#_LT_CONFIG_COMMANDS # Initialize. m4_define([_LT_OUTPUT_LIBTOOL_COMMANDS_INIT], [ # The HP-UX ksh and POSIX shell print the target directory to stdout # if CDPATH is set. (unset CDPATH) >/dev/null 2>&1 && unset CDPATH sed_quote_subst='$sed_quote_subst' double_quote_subst='$double_quote_subst' delay_variable_subst='$delay_variable_subst' _LT_CONFIG_STATUS_DECLARATIONS LTCC='$LTCC' LTCFLAGS='$LTCFLAGS' compiler='$compiler_DEFAULT' # A function that is used when there is no print builtin or printf. func_fallback_echo () { eval 'cat <<_LTECHO_EOF \$[]1 _LTECHO_EOF' } # Quote evaled strings. for var in lt_decl_all_varnames([[ \ ]], lt_decl_quote_varnames); do case \`eval \\\\\$ECHO \\\\""\\\\\$\$var"\\\\"\` in *[[\\\\\\\`\\"\\\$]]*) eval "lt_\$var=\\\\\\"\\\`\\\$ECHO \\"\\\$\$var\\" | \\\$SED \\"\\\$sed_quote_subst\\"\\\`\\\\\\"" ## exclude from sc_prohibit_nested_quotes ;; *) eval "lt_\$var=\\\\\\"\\\$\$var\\\\\\"" ;; esac done # Double-quote double-evaled strings. for var in lt_decl_all_varnames([[ \ ]], lt_decl_dquote_varnames); do case \`eval \\\\\$ECHO \\\\""\\\\\$\$var"\\\\"\` in *[[\\\\\\\`\\"\\\$]]*) eval "lt_\$var=\\\\\\"\\\`\\\$ECHO \\"\\\$\$var\\" | \\\$SED -e \\"\\\$double_quote_subst\\" -e \\"\\\$sed_quote_subst\\" -e \\"\\\$delay_variable_subst\\"\\\`\\\\\\"" ## exclude from sc_prohibit_nested_quotes ;; *) eval "lt_\$var=\\\\\\"\\\$\$var\\\\\\"" ;; esac done _LT_OUTPUT_LIBTOOL_INIT ]) # _LT_GENERATED_FILE_INIT(FILE, [COMMENT]) # ------------------------------------ # Generate a child script FILE with all initialization necessary to # reuse the environment learned by the parent script, and make the # file executable. If COMMENT is supplied, it is inserted after the # '#!' sequence but before initialization text begins. After this # macro, additional text can be appended to FILE to form the body of # the child script. The macro ends with non-zero status if the # file could not be fully written (such as if the disk is full). m4_ifdef([AS_INIT_GENERATED], [m4_defun([_LT_GENERATED_FILE_INIT],[AS_INIT_GENERATED($@)])], [m4_defun([_LT_GENERATED_FILE_INIT], [m4_require([AS_PREPARE])]dnl [m4_pushdef([AS_MESSAGE_LOG_FD])]dnl [lt_write_fail=0 cat >$1 <<_ASEOF || lt_write_fail=1 #! $SHELL # Generated by $as_me. $2 SHELL=\${CONFIG_SHELL-$SHELL} export SHELL _ASEOF cat >>$1 <<\_ASEOF || lt_write_fail=1 AS_SHELL_SANITIZE _AS_PREPARE exec AS_MESSAGE_FD>&1 _ASEOF test 0 = "$lt_write_fail" && chmod +x $1[]dnl m4_popdef([AS_MESSAGE_LOG_FD])])])# _LT_GENERATED_FILE_INIT # LT_OUTPUT # --------- # This macro allows early generation of the libtool script (before # AC_OUTPUT is called), incase it is used in configure for compilation # tests. AC_DEFUN([LT_OUTPUT], [: ${CONFIG_LT=./config.lt} AC_MSG_NOTICE([creating $CONFIG_LT]) _LT_GENERATED_FILE_INIT(["$CONFIG_LT"], [# Run this file to recreate a libtool stub with the current configuration.]) cat >>"$CONFIG_LT" <<\_LTEOF lt_cl_silent=false exec AS_MESSAGE_LOG_FD>>config.log { echo AS_BOX([Running $as_me.]) } >&AS_MESSAGE_LOG_FD lt_cl_help="\ '$as_me' creates a local libtool stub from the current configuration, for use in further configure time tests before the real libtool is generated. Usage: $[0] [[OPTIONS]] -h, --help print this help, then exit -V, --version print version number, then exit -q, --quiet do not print progress messages -d, --debug don't remove temporary files Report bugs to ." lt_cl_version="\ m4_ifset([AC_PACKAGE_NAME], [AC_PACKAGE_NAME ])config.lt[]dnl m4_ifset([AC_PACKAGE_VERSION], [ AC_PACKAGE_VERSION]) configured by $[0], generated by m4_PACKAGE_STRING. Copyright (C) 2011 Free Software Foundation, Inc. This config.lt script is free software; the Free Software Foundation gives unlimited permision to copy, distribute and modify it." while test 0 != $[#] do case $[1] in --version | --v* | -V ) echo "$lt_cl_version"; exit 0 ;; --help | --h* | -h ) echo "$lt_cl_help"; exit 0 ;; --debug | --d* | -d ) debug=: ;; --quiet | --q* | --silent | --s* | -q ) lt_cl_silent=: ;; -*) AC_MSG_ERROR([unrecognized option: $[1] Try '$[0] --help' for more information.]) ;; *) AC_MSG_ERROR([unrecognized argument: $[1] Try '$[0] --help' for more information.]) ;; esac shift done if $lt_cl_silent; then exec AS_MESSAGE_FD>/dev/null fi _LTEOF cat >>"$CONFIG_LT" <<_LTEOF _LT_OUTPUT_LIBTOOL_COMMANDS_INIT _LTEOF cat >>"$CONFIG_LT" <<\_LTEOF AC_MSG_NOTICE([creating $ofile]) _LT_OUTPUT_LIBTOOL_COMMANDS AS_EXIT(0) _LTEOF chmod +x "$CONFIG_LT" # configure is writing to config.log, but config.lt does its own redirection, # appending to config.log, which fails on DOS, as config.log is still kept # open by configure. Here we exec the FD to /dev/null, effectively closing # config.log, so it can be properly (re)opened and appended to by config.lt. lt_cl_success=: test yes = "$silent" && lt_config_lt_args="$lt_config_lt_args --quiet" exec AS_MESSAGE_LOG_FD>/dev/null $SHELL "$CONFIG_LT" $lt_config_lt_args || lt_cl_success=false exec AS_MESSAGE_LOG_FD>>config.log $lt_cl_success || AS_EXIT(1) ])# LT_OUTPUT # _LT_CONFIG(TAG) # --------------- # If TAG is the built-in tag, create an initial libtool script with a # default configuration from the untagged config vars. Otherwise add code # to config.status for appending the configuration named by TAG from the # matching tagged config vars. m4_defun([_LT_CONFIG], [m4_require([_LT_FILEUTILS_DEFAULTS])dnl _LT_CONFIG_SAVE_COMMANDS([ m4_define([_LT_TAG], m4_if([$1], [], [C], [$1]))dnl m4_if(_LT_TAG, [C], [ # See if we are running on zsh, and set the options that allow our # commands through without removal of \ escapes. if test -n "${ZSH_VERSION+set}"; then setopt NO_GLOB_SUBST fi cfgfile=${ofile}T trap "$RM \"$cfgfile\"; exit 1" 1 2 15 $RM "$cfgfile" cat <<_LT_EOF >> "$cfgfile" #! $SHELL # Generated automatically by $as_me ($PACKAGE) $VERSION # Libtool was configured on host `(hostname || uname -n) 2>/dev/null | sed 1q`: # NOTE: Changes made to this file will be lost: look at ltmain.sh. # Provide generalized library-building support services. # Written by Gordon Matzigkeit, 1996 _LT_COPYING _LT_LIBTOOL_TAGS # Configured defaults for sys_lib_dlsearch_path munging. : \${LT_SYS_LIBRARY_PATH="$configure_time_lt_sys_library_path"} # ### BEGIN LIBTOOL CONFIG _LT_LIBTOOL_CONFIG_VARS _LT_LIBTOOL_TAG_VARS # ### END LIBTOOL CONFIG _LT_EOF cat <<'_LT_EOF' >> "$cfgfile" # ### BEGIN FUNCTIONS SHARED WITH CONFIGURE _LT_PREPARE_MUNGE_PATH_LIST _LT_PREPARE_CC_BASENAME # ### END FUNCTIONS SHARED WITH CONFIGURE _LT_EOF case $host_os in aix3*) cat <<\_LT_EOF >> "$cfgfile" # AIX sometimes has problems with the GCC collect2 program. For some # reason, if we set the COLLECT_NAMES environment variable, the problems # vanish in a puff of smoke. if test set != "${COLLECT_NAMES+set}"; then COLLECT_NAMES= export COLLECT_NAMES fi _LT_EOF ;; esac _LT_PROG_LTMAIN # We use sed instead of cat because bash on DJGPP gets confused if # if finds mixed CR/LF and LF-only lines. Since sed operates in # text mode, it properly converts lines to CR/LF. This bash problem # is reportedly fixed, but why not run on old versions too? sed '$q' "$ltmain" >> "$cfgfile" \ || (rm -f "$cfgfile"; exit 1) mv -f "$cfgfile" "$ofile" || (rm -f "$ofile" && cp "$cfgfile" "$ofile" && rm -f "$cfgfile") chmod +x "$ofile" ], [cat <<_LT_EOF >> "$ofile" dnl Unfortunately we have to use $1 here, since _LT_TAG is not expanded dnl in a comment (ie after a #). # ### BEGIN LIBTOOL TAG CONFIG: $1 _LT_LIBTOOL_TAG_VARS(_LT_TAG) # ### END LIBTOOL TAG CONFIG: $1 _LT_EOF ])dnl /m4_if ], [m4_if([$1], [], [ PACKAGE='$PACKAGE' VERSION='$VERSION' RM='$RM' ofile='$ofile'], []) ])dnl /_LT_CONFIG_SAVE_COMMANDS ])# _LT_CONFIG # LT_SUPPORTED_TAG(TAG) # --------------------- # Trace this macro to discover what tags are supported by the libtool # --tag option, using: # autoconf --trace 'LT_SUPPORTED_TAG:$1' AC_DEFUN([LT_SUPPORTED_TAG], []) # C support is built-in for now m4_define([_LT_LANG_C_enabled], []) m4_define([_LT_TAGS], []) # LT_LANG(LANG) # ------------- # Enable libtool support for the given language if not already enabled. AC_DEFUN([LT_LANG], [AC_BEFORE([$0], [LT_OUTPUT])dnl m4_case([$1], [C], [_LT_LANG(C)], [C++], [_LT_LANG(CXX)], [Go], [_LT_LANG(GO)], [Java], [_LT_LANG(GCJ)], [Fortran 77], [_LT_LANG(F77)], [Fortran], [_LT_LANG(FC)], [Windows Resource], [_LT_LANG(RC)], [m4_ifdef([_LT_LANG_]$1[_CONFIG], [_LT_LANG($1)], [m4_fatal([$0: unsupported language: "$1"])])])dnl ])# LT_LANG # _LT_LANG(LANGNAME) # ------------------ m4_defun([_LT_LANG], [m4_ifdef([_LT_LANG_]$1[_enabled], [], [LT_SUPPORTED_TAG([$1])dnl m4_append([_LT_TAGS], [$1 ])dnl m4_define([_LT_LANG_]$1[_enabled], [])dnl _LT_LANG_$1_CONFIG($1)])dnl ])# _LT_LANG m4_ifndef([AC_PROG_GO], [ ############################################################ # NOTE: This macro has been submitted for inclusion into # # GNU Autoconf as AC_PROG_GO. When it is available in # # a released version of Autoconf we should remove this # # macro and use it instead. # ############################################################ m4_defun([AC_PROG_GO], [AC_LANG_PUSH(Go)dnl AC_ARG_VAR([GOC], [Go compiler command])dnl AC_ARG_VAR([GOFLAGS], [Go compiler flags])dnl _AC_ARG_VAR_LDFLAGS()dnl AC_CHECK_TOOL(GOC, gccgo) if test -z "$GOC"; then if test -n "$ac_tool_prefix"; then AC_CHECK_PROG(GOC, [${ac_tool_prefix}gccgo], [${ac_tool_prefix}gccgo]) fi fi if test -z "$GOC"; then AC_CHECK_PROG(GOC, gccgo, gccgo, false) fi ])#m4_defun ])#m4_ifndef # _LT_LANG_DEFAULT_CONFIG # ----------------------- m4_defun([_LT_LANG_DEFAULT_CONFIG], [AC_PROVIDE_IFELSE([AC_PROG_CXX], [LT_LANG(CXX)], [m4_define([AC_PROG_CXX], defn([AC_PROG_CXX])[LT_LANG(CXX)])]) AC_PROVIDE_IFELSE([AC_PROG_F77], [LT_LANG(F77)], [m4_define([AC_PROG_F77], defn([AC_PROG_F77])[LT_LANG(F77)])]) AC_PROVIDE_IFELSE([AC_PROG_FC], [LT_LANG(FC)], [m4_define([AC_PROG_FC], defn([AC_PROG_FC])[LT_LANG(FC)])]) dnl The call to [A][M_PROG_GCJ] is quoted like that to stop aclocal dnl pulling things in needlessly. AC_PROVIDE_IFELSE([AC_PROG_GCJ], [LT_LANG(GCJ)], [AC_PROVIDE_IFELSE([A][M_PROG_GCJ], [LT_LANG(GCJ)], [AC_PROVIDE_IFELSE([LT_PROG_GCJ], [LT_LANG(GCJ)], [m4_ifdef([AC_PROG_GCJ], [m4_define([AC_PROG_GCJ], defn([AC_PROG_GCJ])[LT_LANG(GCJ)])]) m4_ifdef([A][M_PROG_GCJ], [m4_define([A][M_PROG_GCJ], defn([A][M_PROG_GCJ])[LT_LANG(GCJ)])]) m4_ifdef([LT_PROG_GCJ], [m4_define([LT_PROG_GCJ], defn([LT_PROG_GCJ])[LT_LANG(GCJ)])])])])]) AC_PROVIDE_IFELSE([AC_PROG_GO], [LT_LANG(GO)], [m4_define([AC_PROG_GO], defn([AC_PROG_GO])[LT_LANG(GO)])]) AC_PROVIDE_IFELSE([LT_PROG_RC], [LT_LANG(RC)], [m4_define([LT_PROG_RC], defn([LT_PROG_RC])[LT_LANG(RC)])]) ])# _LT_LANG_DEFAULT_CONFIG # Obsolete macros: AU_DEFUN([AC_LIBTOOL_CXX], [LT_LANG(C++)]) AU_DEFUN([AC_LIBTOOL_F77], [LT_LANG(Fortran 77)]) AU_DEFUN([AC_LIBTOOL_FC], [LT_LANG(Fortran)]) AU_DEFUN([AC_LIBTOOL_GCJ], [LT_LANG(Java)]) AU_DEFUN([AC_LIBTOOL_RC], [LT_LANG(Windows Resource)]) dnl aclocal-1.4 backwards compatibility: dnl AC_DEFUN([AC_LIBTOOL_CXX], []) dnl AC_DEFUN([AC_LIBTOOL_F77], []) dnl AC_DEFUN([AC_LIBTOOL_FC], []) dnl AC_DEFUN([AC_LIBTOOL_GCJ], []) dnl AC_DEFUN([AC_LIBTOOL_RC], []) # _LT_TAG_COMPILER # ---------------- m4_defun([_LT_TAG_COMPILER], [AC_REQUIRE([AC_PROG_CC])dnl _LT_DECL([LTCC], [CC], [1], [A C compiler])dnl _LT_DECL([LTCFLAGS], [CFLAGS], [1], [LTCC compiler flags])dnl _LT_TAGDECL([CC], [compiler], [1], [A language specific compiler])dnl _LT_TAGDECL([with_gcc], [GCC], [0], [Is the compiler the GNU compiler?])dnl # If no C compiler was specified, use CC. LTCC=${LTCC-"$CC"} # If no C compiler flags were specified, use CFLAGS. LTCFLAGS=${LTCFLAGS-"$CFLAGS"} # Allow CC to be a program name with arguments. compiler=$CC ])# _LT_TAG_COMPILER # _LT_COMPILER_BOILERPLATE # ------------------------ # Check for compiler boilerplate output or warnings with # the simple compiler test code. m4_defun([_LT_COMPILER_BOILERPLATE], [m4_require([_LT_DECL_SED])dnl ac_outfile=conftest.$ac_objext echo "$lt_simple_compile_test_code" >conftest.$ac_ext eval "$ac_compile" 2>&1 >/dev/null | $SED '/^$/d; /^ *+/d' >conftest.err _lt_compiler_boilerplate=`cat conftest.err` $RM conftest* ])# _LT_COMPILER_BOILERPLATE # _LT_LINKER_BOILERPLATE # ---------------------- # Check for linker boilerplate output or warnings with # the simple link test code. m4_defun([_LT_LINKER_BOILERPLATE], [m4_require([_LT_DECL_SED])dnl ac_outfile=conftest.$ac_objext echo "$lt_simple_link_test_code" >conftest.$ac_ext eval "$ac_link" 2>&1 >/dev/null | $SED '/^$/d; /^ *+/d' >conftest.err _lt_linker_boilerplate=`cat conftest.err` $RM -r conftest* ])# _LT_LINKER_BOILERPLATE # _LT_REQUIRED_DARWIN_CHECKS # ------------------------- m4_defun_once([_LT_REQUIRED_DARWIN_CHECKS],[ case $host_os in rhapsody* | darwin*) AC_CHECK_TOOL([DSYMUTIL], [dsymutil], [:]) AC_CHECK_TOOL([NMEDIT], [nmedit], [:]) AC_CHECK_TOOL([LIPO], [lipo], [:]) AC_CHECK_TOOL([OTOOL], [otool], [:]) AC_CHECK_TOOL([OTOOL64], [otool64], [:]) _LT_DECL([], [DSYMUTIL], [1], [Tool to manipulate archived DWARF debug symbol files on Mac OS X]) _LT_DECL([], [NMEDIT], [1], [Tool to change global to local symbols on Mac OS X]) _LT_DECL([], [LIPO], [1], [Tool to manipulate fat objects and archives on Mac OS X]) _LT_DECL([], [OTOOL], [1], [ldd/readelf like tool for Mach-O binaries on Mac OS X]) _LT_DECL([], [OTOOL64], [1], [ldd/readelf like tool for 64 bit Mach-O binaries on Mac OS X 10.4]) AC_CACHE_CHECK([for -single_module linker flag],[lt_cv_apple_cc_single_mod], [lt_cv_apple_cc_single_mod=no if test -z "$LT_MULTI_MODULE"; then # By default we will add the -single_module flag. You can override # by either setting the environment variable LT_MULTI_MODULE # non-empty at configure time, or by adding -multi_module to the # link flags. rm -rf libconftest.dylib* echo "int foo(void){return 1;}" > conftest.c echo "$LTCC $LTCFLAGS $LDFLAGS -o libconftest.dylib \ -dynamiclib -Wl,-single_module conftest.c" >&AS_MESSAGE_LOG_FD $LTCC $LTCFLAGS $LDFLAGS -o libconftest.dylib \ -dynamiclib -Wl,-single_module conftest.c 2>conftest.err _lt_result=$? # If there is a non-empty error log, and "single_module" # appears in it, assume the flag caused a linker warning if test -s conftest.err && $GREP single_module conftest.err; then cat conftest.err >&AS_MESSAGE_LOG_FD # Otherwise, if the output was created with a 0 exit code from # the compiler, it worked. elif test -f libconftest.dylib && test 0 = "$_lt_result"; then lt_cv_apple_cc_single_mod=yes else cat conftest.err >&AS_MESSAGE_LOG_FD fi rm -rf libconftest.dylib* rm -f conftest.* fi]) AC_CACHE_CHECK([for -exported_symbols_list linker flag], [lt_cv_ld_exported_symbols_list], [lt_cv_ld_exported_symbols_list=no save_LDFLAGS=$LDFLAGS echo "_main" > conftest.sym LDFLAGS="$LDFLAGS -Wl,-exported_symbols_list,conftest.sym" AC_LINK_IFELSE([AC_LANG_PROGRAM([],[])], [lt_cv_ld_exported_symbols_list=yes], [lt_cv_ld_exported_symbols_list=no]) LDFLAGS=$save_LDFLAGS ]) AC_CACHE_CHECK([for -force_load linker flag],[lt_cv_ld_force_load], [lt_cv_ld_force_load=no cat > conftest.c << _LT_EOF int forced_loaded() { return 2;} _LT_EOF echo "$LTCC $LTCFLAGS -c -o conftest.o conftest.c" >&AS_MESSAGE_LOG_FD $LTCC $LTCFLAGS -c -o conftest.o conftest.c 2>&AS_MESSAGE_LOG_FD echo "$AR cru libconftest.a conftest.o" >&AS_MESSAGE_LOG_FD $AR cru libconftest.a conftest.o 2>&AS_MESSAGE_LOG_FD echo "$RANLIB libconftest.a" >&AS_MESSAGE_LOG_FD $RANLIB libconftest.a 2>&AS_MESSAGE_LOG_FD cat > conftest.c << _LT_EOF int main() { return 0;} _LT_EOF echo "$LTCC $LTCFLAGS $LDFLAGS -o conftest conftest.c -Wl,-force_load,./libconftest.a" >&AS_MESSAGE_LOG_FD $LTCC $LTCFLAGS $LDFLAGS -o conftest conftest.c -Wl,-force_load,./libconftest.a 2>conftest.err _lt_result=$? if test -s conftest.err && $GREP force_load conftest.err; then cat conftest.err >&AS_MESSAGE_LOG_FD elif test -f conftest && test 0 = "$_lt_result" && $GREP forced_load conftest >/dev/null 2>&1; then lt_cv_ld_force_load=yes else cat conftest.err >&AS_MESSAGE_LOG_FD fi rm -f conftest.err libconftest.a conftest conftest.c rm -rf conftest.dSYM ]) case $host_os in rhapsody* | darwin1.[[012]]) _lt_dar_allow_undefined='$wl-undefined ${wl}suppress' ;; darwin1.*) _lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;; darwin*) # darwin 5.x on # if running on 10.5 or later, the deployment target defaults # to the OS version, if on x86, and 10.4, the deployment # target defaults to 10.4. Don't you love it? case ${MACOSX_DEPLOYMENT_TARGET-10.0},$host in 10.0,*86*-darwin8*|10.0,*-darwin[[91]]*) _lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;; 10.[[012]][[,.]]*) _lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;; 10.*) _lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;; esac ;; esac if test yes = "$lt_cv_apple_cc_single_mod"; then _lt_dar_single_mod='$single_module' fi if test yes = "$lt_cv_ld_exported_symbols_list"; then _lt_dar_export_syms=' $wl-exported_symbols_list,$output_objdir/$libname-symbols.expsym' else _lt_dar_export_syms='~$NMEDIT -s $output_objdir/$libname-symbols.expsym $lib' fi if test : != "$DSYMUTIL" && test no = "$lt_cv_ld_force_load"; then _lt_dsymutil='~$DSYMUTIL $lib || :' else _lt_dsymutil= fi ;; esac ]) # _LT_DARWIN_LINKER_FEATURES([TAG]) # --------------------------------- # Checks for linker and compiler features on darwin m4_defun([_LT_DARWIN_LINKER_FEATURES], [ m4_require([_LT_REQUIRED_DARWIN_CHECKS]) _LT_TAGVAR(archive_cmds_need_lc, $1)=no _LT_TAGVAR(hardcode_direct, $1)=no _LT_TAGVAR(hardcode_automatic, $1)=yes _LT_TAGVAR(hardcode_shlibpath_var, $1)=unsupported if test yes = "$lt_cv_ld_force_load"; then _LT_TAGVAR(whole_archive_flag_spec, $1)='`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience $wl-force_load,$conv\"; done; func_echo_all \"$new_convenience\"`' m4_case([$1], [F77], [_LT_TAGVAR(compiler_needs_object, $1)=yes], [FC], [_LT_TAGVAR(compiler_needs_object, $1)=yes]) else _LT_TAGVAR(whole_archive_flag_spec, $1)='' fi _LT_TAGVAR(link_all_deplibs, $1)=yes _LT_TAGVAR(allow_undefined_flag, $1)=$_lt_dar_allow_undefined case $cc_basename in ifort*|nagfor*) _lt_dar_can_shared=yes ;; *) _lt_dar_can_shared=$GCC ;; esac if test yes = "$_lt_dar_can_shared"; then output_verbose_link_cmd=func_echo_all _LT_TAGVAR(archive_cmds, $1)="\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$libobjs \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring $_lt_dar_single_mod$_lt_dsymutil" _LT_TAGVAR(module_cmds, $1)="\$CC \$allow_undefined_flag -o \$lib -bundle \$libobjs \$deplibs \$compiler_flags$_lt_dsymutil" _LT_TAGVAR(archive_expsym_cmds, $1)="sed 's|^|_|' < \$export_symbols > \$output_objdir/\$libname-symbols.expsym~\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$libobjs \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring $_lt_dar_single_mod$_lt_dar_export_syms$_lt_dsymutil" _LT_TAGVAR(module_expsym_cmds, $1)="sed -e 's|^|_|' < \$export_symbols > \$output_objdir/\$libname-symbols.expsym~\$CC \$allow_undefined_flag -o \$lib -bundle \$libobjs \$deplibs \$compiler_flags$_lt_dar_export_syms$_lt_dsymutil" m4_if([$1], [CXX], [ if test yes != "$lt_cv_apple_cc_single_mod"; then _LT_TAGVAR(archive_cmds, $1)="\$CC -r -keep_private_externs -nostdlib -o \$lib-master.o \$libobjs~\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$lib-master.o \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring$_lt_dsymutil" _LT_TAGVAR(archive_expsym_cmds, $1)="sed 's|^|_|' < \$export_symbols > \$output_objdir/\$libname-symbols.expsym~\$CC -r -keep_private_externs -nostdlib -o \$lib-master.o \$libobjs~\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$lib-master.o \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring$_lt_dar_export_syms$_lt_dsymutil" fi ],[]) else _LT_TAGVAR(ld_shlibs, $1)=no fi ]) # _LT_SYS_MODULE_PATH_AIX([TAGNAME]) # ---------------------------------- # Links a minimal program and checks the executable # for the system default hardcoded library path. In most cases, # this is /usr/lib:/lib, but when the MPI compilers are used # the location of the communication and MPI libs are included too. # If we don't find anything, use the default library path according # to the aix ld manual. # Store the results from the different compilers for each TAGNAME. # Allow to override them for all tags through lt_cv_aix_libpath. m4_defun([_LT_SYS_MODULE_PATH_AIX], [m4_require([_LT_DECL_SED])dnl if test set = "${lt_cv_aix_libpath+set}"; then aix_libpath=$lt_cv_aix_libpath else AC_CACHE_VAL([_LT_TAGVAR([lt_cv_aix_libpath_], [$1])], [AC_LINK_IFELSE([AC_LANG_PROGRAM],[ lt_aix_libpath_sed='[ /Import File Strings/,/^$/ { /^0/ { s/^0 *\([^ ]*\) *$/\1/ p } }]' _LT_TAGVAR([lt_cv_aix_libpath_], [$1])=`dump -H conftest$ac_exeext 2>/dev/null | $SED -n -e "$lt_aix_libpath_sed"` # Check for a 64-bit object if we didn't find anything. if test -z "$_LT_TAGVAR([lt_cv_aix_libpath_], [$1])"; then _LT_TAGVAR([lt_cv_aix_libpath_], [$1])=`dump -HX64 conftest$ac_exeext 2>/dev/null | $SED -n -e "$lt_aix_libpath_sed"` fi],[]) if test -z "$_LT_TAGVAR([lt_cv_aix_libpath_], [$1])"; then _LT_TAGVAR([lt_cv_aix_libpath_], [$1])=/usr/lib:/lib fi ]) aix_libpath=$_LT_TAGVAR([lt_cv_aix_libpath_], [$1]) fi ])# _LT_SYS_MODULE_PATH_AIX # _LT_SHELL_INIT(ARG) # ------------------- m4_define([_LT_SHELL_INIT], [m4_divert_text([M4SH-INIT], [$1 ])])# _LT_SHELL_INIT # _LT_PROG_ECHO_BACKSLASH # ----------------------- # Find how we can fake an echo command that does not interpret backslash. # In particular, with Autoconf 2.60 or later we add some code to the start # of the generated configure script that will find a shell with a builtin # printf (that we can use as an echo command). m4_defun([_LT_PROG_ECHO_BACKSLASH], [ECHO='\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\' ECHO=$ECHO$ECHO$ECHO$ECHO$ECHO ECHO=$ECHO$ECHO$ECHO$ECHO$ECHO$ECHO AC_MSG_CHECKING([how to print strings]) # Test print first, because it will be a builtin if present. if test "X`( print -r -- -n ) 2>/dev/null`" = X-n && \ test "X`print -r -- $ECHO 2>/dev/null`" = "X$ECHO"; then ECHO='print -r --' elif test "X`printf %s $ECHO 2>/dev/null`" = "X$ECHO"; then ECHO='printf %s\n' else # Use this function as a fallback that always works. func_fallback_echo () { eval 'cat <<_LTECHO_EOF $[]1 _LTECHO_EOF' } ECHO='func_fallback_echo' fi # func_echo_all arg... # Invoke $ECHO with all args, space-separated. func_echo_all () { $ECHO "$*" } case $ECHO in printf*) AC_MSG_RESULT([printf]) ;; print*) AC_MSG_RESULT([print -r]) ;; *) AC_MSG_RESULT([cat]) ;; esac m4_ifdef([_AS_DETECT_SUGGESTED], [_AS_DETECT_SUGGESTED([ test -n "${ZSH_VERSION+set}${BASH_VERSION+set}" || ( ECHO='\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\' ECHO=$ECHO$ECHO$ECHO$ECHO$ECHO ECHO=$ECHO$ECHO$ECHO$ECHO$ECHO$ECHO PATH=/empty FPATH=/empty; export PATH FPATH test "X`printf %s $ECHO`" = "X$ECHO" \ || test "X`print -r -- $ECHO`" = "X$ECHO" )])]) _LT_DECL([], [SHELL], [1], [Shell to use when invoking shell scripts]) _LT_DECL([], [ECHO], [1], [An echo program that protects backslashes]) ])# _LT_PROG_ECHO_BACKSLASH # _LT_WITH_SYSROOT # ---------------- AC_DEFUN([_LT_WITH_SYSROOT], [AC_MSG_CHECKING([for sysroot]) AC_ARG_WITH([sysroot], [AS_HELP_STRING([--with-sysroot@<:@=DIR@:>@], [Search for dependent libraries within DIR (or the compiler's sysroot if not specified).])], [], [with_sysroot=no]) dnl lt_sysroot will always be passed unquoted. We quote it here dnl in case the user passed a directory name. lt_sysroot= case $with_sysroot in #( yes) if test yes = "$GCC"; then lt_sysroot=`$CC --print-sysroot 2>/dev/null` fi ;; #( /*) lt_sysroot=`echo "$with_sysroot" | sed -e "$sed_quote_subst"` ;; #( no|'') ;; #( *) AC_MSG_RESULT([$with_sysroot]) AC_MSG_ERROR([The sysroot must be an absolute path.]) ;; esac AC_MSG_RESULT([${lt_sysroot:-no}]) _LT_DECL([], [lt_sysroot], [0], [The root where to search for ]dnl [dependent libraries, and where our libraries should be installed.])]) # _LT_ENABLE_LOCK # --------------- m4_defun([_LT_ENABLE_LOCK], [AC_ARG_ENABLE([libtool-lock], [AS_HELP_STRING([--disable-libtool-lock], [avoid locking (might break parallel builds)])]) test no = "$enable_libtool_lock" || enable_libtool_lock=yes # Some flags need to be propagated to the compiler or linker for good # libtool support. case $host in ia64-*-hpux*) # Find out what ABI is being produced by ac_compile, and set mode # options accordingly. echo 'int i;' > conftest.$ac_ext if AC_TRY_EVAL(ac_compile); then case `/usr/bin/file conftest.$ac_objext` in *ELF-32*) HPUX_IA64_MODE=32 ;; *ELF-64*) HPUX_IA64_MODE=64 ;; esac fi rm -rf conftest* ;; *-*-irix6*) # Find out what ABI is being produced by ac_compile, and set linker # options accordingly. echo '[#]line '$LINENO' "configure"' > conftest.$ac_ext if AC_TRY_EVAL(ac_compile); then if test yes = "$lt_cv_prog_gnu_ld"; then case `/usr/bin/file conftest.$ac_objext` in *32-bit*) LD="${LD-ld} -melf32bsmip" ;; *N32*) LD="${LD-ld} -melf32bmipn32" ;; *64-bit*) LD="${LD-ld} -melf64bmip" ;; esac else case `/usr/bin/file conftest.$ac_objext` in *32-bit*) LD="${LD-ld} -32" ;; *N32*) LD="${LD-ld} -n32" ;; *64-bit*) LD="${LD-ld} -64" ;; esac fi fi rm -rf conftest* ;; mips64*-*linux*) # Find out what ABI is being produced by ac_compile, and set linker # options accordingly. echo '[#]line '$LINENO' "configure"' > conftest.$ac_ext if AC_TRY_EVAL(ac_compile); then emul=elf case `/usr/bin/file conftest.$ac_objext` in *32-bit*) emul="${emul}32" ;; *64-bit*) emul="${emul}64" ;; esac case `/usr/bin/file conftest.$ac_objext` in *MSB*) emul="${emul}btsmip" ;; *LSB*) emul="${emul}ltsmip" ;; esac case `/usr/bin/file conftest.$ac_objext` in *N32*) emul="${emul}n32" ;; esac LD="${LD-ld} -m $emul" fi rm -rf conftest* ;; x86_64-*kfreebsd*-gnu|x86_64-*linux*|powerpc*-*linux*| \ s390*-*linux*|s390*-*tpf*|sparc*-*linux*) # Find out what ABI is being produced by ac_compile, and set linker # options accordingly. Note that the listed cases only cover the # situations where additional linker options are needed (such as when # doing 32-bit compilation for a host where ld defaults to 64-bit, or # vice versa); the common cases where no linker options are needed do # not appear in the list. echo 'int i;' > conftest.$ac_ext if AC_TRY_EVAL(ac_compile); then case `/usr/bin/file conftest.o` in *32-bit*) case $host in x86_64-*kfreebsd*-gnu) LD="${LD-ld} -m elf_i386_fbsd" ;; x86_64-*linux*) case `/usr/bin/file conftest.o` in *x86-64*) LD="${LD-ld} -m elf32_x86_64" ;; *) LD="${LD-ld} -m elf_i386" ;; esac ;; powerpc64le-*linux*) LD="${LD-ld} -m elf32lppclinux" ;; powerpc64-*linux*) LD="${LD-ld} -m elf32ppclinux" ;; s390x-*linux*) LD="${LD-ld} -m elf_s390" ;; sparc64-*linux*) LD="${LD-ld} -m elf32_sparc" ;; esac ;; *64-bit*) case $host in x86_64-*kfreebsd*-gnu) LD="${LD-ld} -m elf_x86_64_fbsd" ;; x86_64-*linux*) LD="${LD-ld} -m elf_x86_64" ;; powerpcle-*linux*) LD="${LD-ld} -m elf64lppc" ;; powerpc-*linux*) LD="${LD-ld} -m elf64ppc" ;; s390*-*linux*|s390*-*tpf*) LD="${LD-ld} -m elf64_s390" ;; sparc*-*linux*) LD="${LD-ld} -m elf64_sparc" ;; esac ;; esac fi rm -rf conftest* ;; *-*-sco3.2v5*) # On SCO OpenServer 5, we need -belf to get full-featured binaries. SAVE_CFLAGS=$CFLAGS CFLAGS="$CFLAGS -belf" AC_CACHE_CHECK([whether the C compiler needs -belf], lt_cv_cc_needs_belf, [AC_LANG_PUSH(C) AC_LINK_IFELSE([AC_LANG_PROGRAM([[]],[[]])],[lt_cv_cc_needs_belf=yes],[lt_cv_cc_needs_belf=no]) AC_LANG_POP]) if test yes != "$lt_cv_cc_needs_belf"; then # this is probably gcc 2.8.0, egcs 1.0 or newer; no need for -belf CFLAGS=$SAVE_CFLAGS fi ;; *-*solaris*) # Find out what ABI is being produced by ac_compile, and set linker # options accordingly. echo 'int i;' > conftest.$ac_ext if AC_TRY_EVAL(ac_compile); then case `/usr/bin/file conftest.o` in *64-bit*) case $lt_cv_prog_gnu_ld in yes*) case $host in i?86-*-solaris*|x86_64-*-solaris*) LD="${LD-ld} -m elf_x86_64" ;; sparc*-*-solaris*) LD="${LD-ld} -m elf64_sparc" ;; esac # GNU ld 2.21 introduced _sol2 emulations. Use them if available. if ${LD-ld} -V | grep _sol2 >/dev/null 2>&1; then LD=${LD-ld}_sol2 fi ;; *) if ${LD-ld} -64 -r -o conftest2.o conftest.o >/dev/null 2>&1; then LD="${LD-ld} -64" fi ;; esac ;; esac fi rm -rf conftest* ;; esac need_locks=$enable_libtool_lock ])# _LT_ENABLE_LOCK # _LT_PROG_AR # ----------- m4_defun([_LT_PROG_AR], [AC_CHECK_TOOLS(AR, [ar], false) : ${AR=ar} : ${AR_FLAGS=cru} _LT_DECL([], [AR], [1], [The archiver]) _LT_DECL([], [AR_FLAGS], [1], [Flags to create an archive]) AC_CACHE_CHECK([for archiver @FILE support], [lt_cv_ar_at_file], [lt_cv_ar_at_file=no AC_COMPILE_IFELSE([AC_LANG_PROGRAM], [echo conftest.$ac_objext > conftest.lst lt_ar_try='$AR $AR_FLAGS libconftest.a @conftest.lst >&AS_MESSAGE_LOG_FD' AC_TRY_EVAL([lt_ar_try]) if test 0 -eq "$ac_status"; then # Ensure the archiver fails upon bogus file names. rm -f conftest.$ac_objext libconftest.a AC_TRY_EVAL([lt_ar_try]) if test 0 -ne "$ac_status"; then lt_cv_ar_at_file=@ fi fi rm -f conftest.* libconftest.a ]) ]) if test no = "$lt_cv_ar_at_file"; then archiver_list_spec= else archiver_list_spec=$lt_cv_ar_at_file fi _LT_DECL([], [archiver_list_spec], [1], [How to feed a file listing to the archiver]) ])# _LT_PROG_AR # _LT_CMD_OLD_ARCHIVE # ------------------- m4_defun([_LT_CMD_OLD_ARCHIVE], [_LT_PROG_AR AC_CHECK_TOOL(STRIP, strip, :) test -z "$STRIP" && STRIP=: _LT_DECL([], [STRIP], [1], [A symbol stripping program]) AC_CHECK_TOOL(RANLIB, ranlib, :) test -z "$RANLIB" && RANLIB=: _LT_DECL([], [RANLIB], [1], [Commands used to install an old-style archive]) # Determine commands to create old-style static archives. old_archive_cmds='$AR $AR_FLAGS $oldlib$oldobjs' old_postinstall_cmds='chmod 644 $oldlib' old_postuninstall_cmds= if test -n "$RANLIB"; then case $host_os in bitrig* | openbsd*) old_postinstall_cmds="$old_postinstall_cmds~\$RANLIB -t \$tool_oldlib" ;; *) old_postinstall_cmds="$old_postinstall_cmds~\$RANLIB \$tool_oldlib" ;; esac old_archive_cmds="$old_archive_cmds~\$RANLIB \$tool_oldlib" fi case $host_os in darwin*) lock_old_archive_extraction=yes ;; *) lock_old_archive_extraction=no ;; esac _LT_DECL([], [old_postinstall_cmds], [2]) _LT_DECL([], [old_postuninstall_cmds], [2]) _LT_TAGDECL([], [old_archive_cmds], [2], [Commands used to build an old-style archive]) _LT_DECL([], [lock_old_archive_extraction], [0], [Whether to use a lock for old archive extraction]) ])# _LT_CMD_OLD_ARCHIVE # _LT_COMPILER_OPTION(MESSAGE, VARIABLE-NAME, FLAGS, # [OUTPUT-FILE], [ACTION-SUCCESS], [ACTION-FAILURE]) # ---------------------------------------------------------------- # Check whether the given compiler option works AC_DEFUN([_LT_COMPILER_OPTION], [m4_require([_LT_FILEUTILS_DEFAULTS])dnl m4_require([_LT_DECL_SED])dnl AC_CACHE_CHECK([$1], [$2], [$2=no m4_if([$4], , [ac_outfile=conftest.$ac_objext], [ac_outfile=$4]) echo "$lt_simple_compile_test_code" > conftest.$ac_ext lt_compiler_flag="$3" ## exclude from sc_useless_quotes_in_assignment # Insert the option either (1) after the last *FLAGS variable, or # (2) before a word containing "conftest.", or (3) at the end. # Note that $ac_compile itself does not contain backslashes and begins # with a dollar sign (not a hyphen), so the echo should work correctly. # The option is referenced via a variable to avoid confusing sed. lt_compile=`echo "$ac_compile" | $SED \ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [[^ ]]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` (eval echo "\"\$as_me:$LINENO: $lt_compile\"" >&AS_MESSAGE_LOG_FD) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&AS_MESSAGE_LOG_FD echo "$as_me:$LINENO: \$? = $ac_status" >&AS_MESSAGE_LOG_FD if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings other than the usual output. $ECHO "$_lt_compiler_boilerplate" | $SED '/^$/d' >conftest.exp $SED '/^$/d; /^ *+/d' conftest.err >conftest.er2 if test ! -s conftest.er2 || diff conftest.exp conftest.er2 >/dev/null; then $2=yes fi fi $RM conftest* ]) if test yes = "[$]$2"; then m4_if([$5], , :, [$5]) else m4_if([$6], , :, [$6]) fi ])# _LT_COMPILER_OPTION # Old name: AU_ALIAS([AC_LIBTOOL_COMPILER_OPTION], [_LT_COMPILER_OPTION]) dnl aclocal-1.4 backwards compatibility: dnl AC_DEFUN([AC_LIBTOOL_COMPILER_OPTION], []) # _LT_LINKER_OPTION(MESSAGE, VARIABLE-NAME, FLAGS, # [ACTION-SUCCESS], [ACTION-FAILURE]) # ---------------------------------------------------- # Check whether the given linker option works AC_DEFUN([_LT_LINKER_OPTION], [m4_require([_LT_FILEUTILS_DEFAULTS])dnl m4_require([_LT_DECL_SED])dnl AC_CACHE_CHECK([$1], [$2], [$2=no save_LDFLAGS=$LDFLAGS LDFLAGS="$LDFLAGS $3" echo "$lt_simple_link_test_code" > conftest.$ac_ext if (eval $ac_link 2>conftest.err) && test -s conftest$ac_exeext; then # The linker can only warn and ignore the option if not recognized # So say no if there are warnings if test -s conftest.err; then # Append any errors to the config.log. cat conftest.err 1>&AS_MESSAGE_LOG_FD $ECHO "$_lt_linker_boilerplate" | $SED '/^$/d' > conftest.exp $SED '/^$/d; /^ *+/d' conftest.err >conftest.er2 if diff conftest.exp conftest.er2 >/dev/null; then $2=yes fi else $2=yes fi fi $RM -r conftest* LDFLAGS=$save_LDFLAGS ]) if test yes = "[$]$2"; then m4_if([$4], , :, [$4]) else m4_if([$5], , :, [$5]) fi ])# _LT_LINKER_OPTION # Old name: AU_ALIAS([AC_LIBTOOL_LINKER_OPTION], [_LT_LINKER_OPTION]) dnl aclocal-1.4 backwards compatibility: dnl AC_DEFUN([AC_LIBTOOL_LINKER_OPTION], []) # LT_CMD_MAX_LEN #--------------- AC_DEFUN([LT_CMD_MAX_LEN], [AC_REQUIRE([AC_CANONICAL_HOST])dnl # find the maximum length of command line arguments AC_MSG_CHECKING([the maximum length of command line arguments]) AC_CACHE_VAL([lt_cv_sys_max_cmd_len], [dnl i=0 teststring=ABCD case $build_os in msdosdjgpp*) # On DJGPP, this test can blow up pretty badly due to problems in libc # (any single argument exceeding 2000 bytes causes a buffer overrun # during glob expansion). Even if it were fixed, the result of this # check would be larger than it should be. lt_cv_sys_max_cmd_len=12288; # 12K is about right ;; gnu*) # Under GNU Hurd, this test is not required because there is # no limit to the length of command line arguments. # Libtool will interpret -1 as no limit whatsoever lt_cv_sys_max_cmd_len=-1; ;; cygwin* | mingw* | cegcc*) # On Win9x/ME, this test blows up -- it succeeds, but takes # about 5 minutes as the teststring grows exponentially. # Worse, since 9x/ME are not pre-emptively multitasking, # you end up with a "frozen" computer, even though with patience # the test eventually succeeds (with a max line length of 256k). # Instead, let's just punt: use the minimum linelength reported by # all of the supported platforms: 8192 (on NT/2K/XP). lt_cv_sys_max_cmd_len=8192; ;; mint*) # On MiNT this can take a long time and run out of memory. lt_cv_sys_max_cmd_len=8192; ;; amigaos*) # On AmigaOS with pdksh, this test takes hours, literally. # So we just punt and use a minimum line length of 8192. lt_cv_sys_max_cmd_len=8192; ;; bitrig* | darwin* | dragonfly* | freebsd* | netbsd* | openbsd*) # This has been around since 386BSD, at least. Likely further. if test -x /sbin/sysctl; then lt_cv_sys_max_cmd_len=`/sbin/sysctl -n kern.argmax` elif test -x /usr/sbin/sysctl; then lt_cv_sys_max_cmd_len=`/usr/sbin/sysctl -n kern.argmax` else lt_cv_sys_max_cmd_len=65536 # usable default for all BSDs fi # And add a safety zone lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \/ 4` lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \* 3` ;; interix*) # We know the value 262144 and hardcode it with a safety zone (like BSD) lt_cv_sys_max_cmd_len=196608 ;; os2*) # The test takes a long time on OS/2. lt_cv_sys_max_cmd_len=8192 ;; osf*) # Dr. Hans Ekkehard Plesser reports seeing a kernel panic running configure # due to this test when exec_disable_arg_limit is 1 on Tru64. It is not # nice to cause kernel panics so lets avoid the loop below. # First set a reasonable default. lt_cv_sys_max_cmd_len=16384 # if test -x /sbin/sysconfig; then case `/sbin/sysconfig -q proc exec_disable_arg_limit` in *1*) lt_cv_sys_max_cmd_len=-1 ;; esac fi ;; sco3.2v5*) lt_cv_sys_max_cmd_len=102400 ;; sysv5* | sco5v6* | sysv4.2uw2*) kargmax=`grep ARG_MAX /etc/conf/cf.d/stune 2>/dev/null` if test -n "$kargmax"; then lt_cv_sys_max_cmd_len=`echo $kargmax | sed 's/.*[[ ]]//'` else lt_cv_sys_max_cmd_len=32768 fi ;; *) lt_cv_sys_max_cmd_len=`(getconf ARG_MAX) 2> /dev/null` if test -n "$lt_cv_sys_max_cmd_len" && \ test undefined != "$lt_cv_sys_max_cmd_len"; then lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \/ 4` lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \* 3` else # Make teststring a little bigger before we do anything with it. # a 1K string should be a reasonable start. for i in 1 2 3 4 5 6 7 8; do teststring=$teststring$teststring done SHELL=${SHELL-${CONFIG_SHELL-/bin/sh}} # If test is not a shell built-in, we'll probably end up computing a # maximum length that is only half of the actual maximum length, but # we can't tell. while { test X`env echo "$teststring$teststring" 2>/dev/null` \ = "X$teststring$teststring"; } >/dev/null 2>&1 && test 17 != "$i" # 1/2 MB should be enough do i=`expr $i + 1` teststring=$teststring$teststring done # Only check the string length outside the loop. lt_cv_sys_max_cmd_len=`expr "X$teststring" : ".*" 2>&1` teststring= # Add a significant safety factor because C++ compilers can tack on # massive amounts of additional arguments before passing them to the # linker. It appears as though 1/2 is a usable value. lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \/ 2` fi ;; esac ]) if test -n "$lt_cv_sys_max_cmd_len"; then AC_MSG_RESULT($lt_cv_sys_max_cmd_len) else AC_MSG_RESULT(none) fi max_cmd_len=$lt_cv_sys_max_cmd_len _LT_DECL([], [max_cmd_len], [0], [What is the maximum length of a command?]) ])# LT_CMD_MAX_LEN # Old name: AU_ALIAS([AC_LIBTOOL_SYS_MAX_CMD_LEN], [LT_CMD_MAX_LEN]) dnl aclocal-1.4 backwards compatibility: dnl AC_DEFUN([AC_LIBTOOL_SYS_MAX_CMD_LEN], []) # _LT_HEADER_DLFCN # ---------------- m4_defun([_LT_HEADER_DLFCN], [AC_CHECK_HEADERS([dlfcn.h], [], [], [AC_INCLUDES_DEFAULT])dnl ])# _LT_HEADER_DLFCN # _LT_TRY_DLOPEN_SELF (ACTION-IF-TRUE, ACTION-IF-TRUE-W-USCORE, # ACTION-IF-FALSE, ACTION-IF-CROSS-COMPILING) # ---------------------------------------------------------------- m4_defun([_LT_TRY_DLOPEN_SELF], [m4_require([_LT_HEADER_DLFCN])dnl if test yes = "$cross_compiling"; then : [$4] else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF [#line $LINENO "configure" #include "confdefs.h" #if HAVE_DLFCN_H #include #endif #include #ifdef RTLD_GLOBAL # define LT_DLGLOBAL RTLD_GLOBAL #else # ifdef DL_GLOBAL # define LT_DLGLOBAL DL_GLOBAL # else # define LT_DLGLOBAL 0 # endif #endif /* We may have to define LT_DLLAZY_OR_NOW in the command line if we find out it does not work in some platform. */ #ifndef LT_DLLAZY_OR_NOW # ifdef RTLD_LAZY # define LT_DLLAZY_OR_NOW RTLD_LAZY # else # ifdef DL_LAZY # define LT_DLLAZY_OR_NOW DL_LAZY # else # ifdef RTLD_NOW # define LT_DLLAZY_OR_NOW RTLD_NOW # else # ifdef DL_NOW # define LT_DLLAZY_OR_NOW DL_NOW # else # define LT_DLLAZY_OR_NOW 0 # endif # endif # endif # endif #endif /* When -fvisibility=hidden is used, assume the code has been annotated correspondingly for the symbols needed. */ #if defined __GNUC__ && (((__GNUC__ == 3) && (__GNUC_MINOR__ >= 3)) || (__GNUC__ > 3)) int fnord () __attribute__((visibility("default"))); #endif int fnord () { return 42; } int main () { void *self = dlopen (0, LT_DLGLOBAL|LT_DLLAZY_OR_NOW); int status = $lt_dlunknown; if (self) { if (dlsym (self,"fnord")) status = $lt_dlno_uscore; else { if (dlsym( self,"_fnord")) status = $lt_dlneed_uscore; else puts (dlerror ()); } /* dlclose (self); */ } else puts (dlerror ()); return status; }] _LT_EOF if AC_TRY_EVAL(ac_link) && test -s "conftest$ac_exeext" 2>/dev/null; then (./conftest; exit; ) >&AS_MESSAGE_LOG_FD 2>/dev/null lt_status=$? case x$lt_status in x$lt_dlno_uscore) $1 ;; x$lt_dlneed_uscore) $2 ;; x$lt_dlunknown|x*) $3 ;; esac else : # compilation failed $3 fi fi rm -fr conftest* ])# _LT_TRY_DLOPEN_SELF # LT_SYS_DLOPEN_SELF # ------------------ AC_DEFUN([LT_SYS_DLOPEN_SELF], [m4_require([_LT_HEADER_DLFCN])dnl if test yes != "$enable_dlopen"; then enable_dlopen=unknown enable_dlopen_self=unknown enable_dlopen_self_static=unknown else lt_cv_dlopen=no lt_cv_dlopen_libs= case $host_os in beos*) lt_cv_dlopen=load_add_on lt_cv_dlopen_libs= lt_cv_dlopen_self=yes ;; mingw* | pw32* | cegcc*) lt_cv_dlopen=LoadLibrary lt_cv_dlopen_libs= ;; cygwin*) lt_cv_dlopen=dlopen lt_cv_dlopen_libs= ;; darwin*) # if libdl is installed we need to link against it AC_CHECK_LIB([dl], [dlopen], [lt_cv_dlopen=dlopen lt_cv_dlopen_libs=-ldl],[ lt_cv_dlopen=dyld lt_cv_dlopen_libs= lt_cv_dlopen_self=yes ]) ;; tpf*) # Don't try to run any link tests for TPF. We know it's impossible # because TPF is a cross-compiler, and we know how we open DSOs. lt_cv_dlopen=dlopen lt_cv_dlopen_libs= lt_cv_dlopen_self=no ;; *) AC_CHECK_FUNC([shl_load], [lt_cv_dlopen=shl_load], [AC_CHECK_LIB([dld], [shl_load], [lt_cv_dlopen=shl_load lt_cv_dlopen_libs=-ldld], [AC_CHECK_FUNC([dlopen], [lt_cv_dlopen=dlopen], [AC_CHECK_LIB([dl], [dlopen], [lt_cv_dlopen=dlopen lt_cv_dlopen_libs=-ldl], [AC_CHECK_LIB([svld], [dlopen], [lt_cv_dlopen=dlopen lt_cv_dlopen_libs=-lsvld], [AC_CHECK_LIB([dld], [dld_link], [lt_cv_dlopen=dld_link lt_cv_dlopen_libs=-ldld]) ]) ]) ]) ]) ]) ;; esac if test no = "$lt_cv_dlopen"; then enable_dlopen=no else enable_dlopen=yes fi case $lt_cv_dlopen in dlopen) save_CPPFLAGS=$CPPFLAGS test yes = "$ac_cv_header_dlfcn_h" && CPPFLAGS="$CPPFLAGS -DHAVE_DLFCN_H" save_LDFLAGS=$LDFLAGS wl=$lt_prog_compiler_wl eval LDFLAGS=\"\$LDFLAGS $export_dynamic_flag_spec\" save_LIBS=$LIBS LIBS="$lt_cv_dlopen_libs $LIBS" AC_CACHE_CHECK([whether a program can dlopen itself], lt_cv_dlopen_self, [dnl _LT_TRY_DLOPEN_SELF( lt_cv_dlopen_self=yes, lt_cv_dlopen_self=yes, lt_cv_dlopen_self=no, lt_cv_dlopen_self=cross) ]) if test yes = "$lt_cv_dlopen_self"; then wl=$lt_prog_compiler_wl eval LDFLAGS=\"\$LDFLAGS $lt_prog_compiler_static\" AC_CACHE_CHECK([whether a statically linked program can dlopen itself], lt_cv_dlopen_self_static, [dnl _LT_TRY_DLOPEN_SELF( lt_cv_dlopen_self_static=yes, lt_cv_dlopen_self_static=yes, lt_cv_dlopen_self_static=no, lt_cv_dlopen_self_static=cross) ]) fi CPPFLAGS=$save_CPPFLAGS LDFLAGS=$save_LDFLAGS LIBS=$save_LIBS ;; esac case $lt_cv_dlopen_self in yes|no) enable_dlopen_self=$lt_cv_dlopen_self ;; *) enable_dlopen_self=unknown ;; esac case $lt_cv_dlopen_self_static in yes|no) enable_dlopen_self_static=$lt_cv_dlopen_self_static ;; *) enable_dlopen_self_static=unknown ;; esac fi _LT_DECL([dlopen_support], [enable_dlopen], [0], [Whether dlopen is supported]) _LT_DECL([dlopen_self], [enable_dlopen_self], [0], [Whether dlopen of programs is supported]) _LT_DECL([dlopen_self_static], [enable_dlopen_self_static], [0], [Whether dlopen of statically linked programs is supported]) ])# LT_SYS_DLOPEN_SELF # Old name: AU_ALIAS([AC_LIBTOOL_DLOPEN_SELF], [LT_SYS_DLOPEN_SELF]) dnl aclocal-1.4 backwards compatibility: dnl AC_DEFUN([AC_LIBTOOL_DLOPEN_SELF], []) # _LT_COMPILER_C_O([TAGNAME]) # --------------------------- # Check to see if options -c and -o are simultaneously supported by compiler. # This macro does not hard code the compiler like AC_PROG_CC_C_O. m4_defun([_LT_COMPILER_C_O], [m4_require([_LT_DECL_SED])dnl m4_require([_LT_FILEUTILS_DEFAULTS])dnl m4_require([_LT_TAG_COMPILER])dnl AC_CACHE_CHECK([if $compiler supports -c -o file.$ac_objext], [_LT_TAGVAR(lt_cv_prog_compiler_c_o, $1)], [_LT_TAGVAR(lt_cv_prog_compiler_c_o, $1)=no $RM -r conftest 2>/dev/null mkdir conftest cd conftest mkdir out echo "$lt_simple_compile_test_code" > conftest.$ac_ext lt_compiler_flag="-o out/conftest2.$ac_objext" # Insert the option either (1) after the last *FLAGS variable, or # (2) before a word containing "conftest.", or (3) at the end. # Note that $ac_compile itself does not contain backslashes and begins # with a dollar sign (not a hyphen), so the echo should work correctly. lt_compile=`echo "$ac_compile" | $SED \ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [[^ ]]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` (eval echo "\"\$as_me:$LINENO: $lt_compile\"" >&AS_MESSAGE_LOG_FD) (eval "$lt_compile" 2>out/conftest.err) ac_status=$? cat out/conftest.err >&AS_MESSAGE_LOG_FD echo "$as_me:$LINENO: \$? = $ac_status" >&AS_MESSAGE_LOG_FD if (exit $ac_status) && test -s out/conftest2.$ac_objext then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings $ECHO "$_lt_compiler_boilerplate" | $SED '/^$/d' > out/conftest.exp $SED '/^$/d; /^ *+/d' out/conftest.err >out/conftest.er2 if test ! -s out/conftest.er2 || diff out/conftest.exp out/conftest.er2 >/dev/null; then _LT_TAGVAR(lt_cv_prog_compiler_c_o, $1)=yes fi fi chmod u+w . 2>&AS_MESSAGE_LOG_FD $RM conftest* # SGI C++ compiler will create directory out/ii_files/ for # template instantiation test -d out/ii_files && $RM out/ii_files/* && rmdir out/ii_files $RM out/* && rmdir out cd .. $RM -r conftest $RM conftest* ]) _LT_TAGDECL([compiler_c_o], [lt_cv_prog_compiler_c_o], [1], [Does compiler simultaneously support -c and -o options?]) ])# _LT_COMPILER_C_O # _LT_COMPILER_FILE_LOCKS([TAGNAME]) # ---------------------------------- # Check to see if we can do hard links to lock some files if needed m4_defun([_LT_COMPILER_FILE_LOCKS], [m4_require([_LT_ENABLE_LOCK])dnl m4_require([_LT_FILEUTILS_DEFAULTS])dnl _LT_COMPILER_C_O([$1]) hard_links=nottested if test no = "$_LT_TAGVAR(lt_cv_prog_compiler_c_o, $1)" && test no != "$need_locks"; then # do not overwrite the value of need_locks provided by the user AC_MSG_CHECKING([if we can lock with hard links]) hard_links=yes $RM conftest* ln conftest.a conftest.b 2>/dev/null && hard_links=no touch conftest.a ln conftest.a conftest.b 2>&5 || hard_links=no ln conftest.a conftest.b 2>/dev/null && hard_links=no AC_MSG_RESULT([$hard_links]) if test no = "$hard_links"; then AC_MSG_WARN(['$CC' does not support '-c -o', so 'make -j' may be unsafe]) need_locks=warn fi else need_locks=no fi _LT_DECL([], [need_locks], [1], [Must we lock files when doing compilation?]) ])# _LT_COMPILER_FILE_LOCKS # _LT_CHECK_OBJDIR # ---------------- m4_defun([_LT_CHECK_OBJDIR], [AC_CACHE_CHECK([for objdir], [lt_cv_objdir], [rm -f .libs 2>/dev/null mkdir .libs 2>/dev/null if test -d .libs; then lt_cv_objdir=.libs else # MS-DOS does not allow filenames that begin with a dot. lt_cv_objdir=_libs fi rmdir .libs 2>/dev/null]) objdir=$lt_cv_objdir _LT_DECL([], [objdir], [0], [The name of the directory that contains temporary libtool files])dnl m4_pattern_allow([LT_OBJDIR])dnl AC_DEFINE_UNQUOTED([LT_OBJDIR], "$lt_cv_objdir/", [Define to the sub-directory where libtool stores uninstalled libraries.]) ])# _LT_CHECK_OBJDIR # _LT_LINKER_HARDCODE_LIBPATH([TAGNAME]) # -------------------------------------- # Check hardcoding attributes. m4_defun([_LT_LINKER_HARDCODE_LIBPATH], [AC_MSG_CHECKING([how to hardcode library paths into programs]) _LT_TAGVAR(hardcode_action, $1)= if test -n "$_LT_TAGVAR(hardcode_libdir_flag_spec, $1)" || test -n "$_LT_TAGVAR(runpath_var, $1)" || test yes = "$_LT_TAGVAR(hardcode_automatic, $1)"; then # We can hardcode non-existent directories. if test no != "$_LT_TAGVAR(hardcode_direct, $1)" && # If the only mechanism to avoid hardcoding is shlibpath_var, we # have to relink, otherwise we might link with an installed library # when we should be linking with a yet-to-be-installed one ## test no != "$_LT_TAGVAR(hardcode_shlibpath_var, $1)" && test no != "$_LT_TAGVAR(hardcode_minus_L, $1)"; then # Linking always hardcodes the temporary library directory. _LT_TAGVAR(hardcode_action, $1)=relink else # We can link without hardcoding, and we can hardcode nonexisting dirs. _LT_TAGVAR(hardcode_action, $1)=immediate fi else # We cannot hardcode anything, or else we can only hardcode existing # directories. _LT_TAGVAR(hardcode_action, $1)=unsupported fi AC_MSG_RESULT([$_LT_TAGVAR(hardcode_action, $1)]) if test relink = "$_LT_TAGVAR(hardcode_action, $1)" || test yes = "$_LT_TAGVAR(inherit_rpath, $1)"; then # Fast installation is not supported enable_fast_install=no elif test yes = "$shlibpath_overrides_runpath" || test no = "$enable_shared"; then # Fast installation is not necessary enable_fast_install=needless fi _LT_TAGDECL([], [hardcode_action], [0], [How to hardcode a shared library path into an executable]) ])# _LT_LINKER_HARDCODE_LIBPATH # _LT_CMD_STRIPLIB # ---------------- m4_defun([_LT_CMD_STRIPLIB], [m4_require([_LT_DECL_EGREP]) striplib= old_striplib= AC_MSG_CHECKING([whether stripping libraries is possible]) if test -n "$STRIP" && $STRIP -V 2>&1 | $GREP "GNU strip" >/dev/null; then test -z "$old_striplib" && old_striplib="$STRIP --strip-debug" test -z "$striplib" && striplib="$STRIP --strip-unneeded" AC_MSG_RESULT([yes]) else # FIXME - insert some real tests, host_os isn't really good enough case $host_os in darwin*) if test -n "$STRIP"; then striplib="$STRIP -x" old_striplib="$STRIP -S" AC_MSG_RESULT([yes]) else AC_MSG_RESULT([no]) fi ;; *) AC_MSG_RESULT([no]) ;; esac fi _LT_DECL([], [old_striplib], [1], [Commands to strip libraries]) _LT_DECL([], [striplib], [1]) ])# _LT_CMD_STRIPLIB # _LT_PREPARE_MUNGE_PATH_LIST # --------------------------- # Make sure func_munge_path_list() is defined correctly. m4_defun([_LT_PREPARE_MUNGE_PATH_LIST], [[# func_munge_path_list VARIABLE PATH # ----------------------------------- # VARIABLE is name of variable containing _space_ separated list of # directories to be munged by the contents of PATH, which is string # having a format: # "DIR[:DIR]:" # string "DIR[ DIR]" will be prepended to VARIABLE # ":DIR[:DIR]" # string "DIR[ DIR]" will be appended to VARIABLE # "DIRP[:DIRP]::[DIRA:]DIRA" # string "DIRP[ DIRP]" will be prepended to VARIABLE and string # "DIRA[ DIRA]" will be appended to VARIABLE # "DIR[:DIR]" # VARIABLE will be replaced by "DIR[ DIR]" func_munge_path_list () { case x@S|@2 in x) ;; *:) eval @S|@1=\"`$ECHO @S|@2 | $SED 's/:/ /g'` \@S|@@S|@1\" ;; x:*) eval @S|@1=\"\@S|@@S|@1 `$ECHO @S|@2 | $SED 's/:/ /g'`\" ;; *::*) eval @S|@1=\"\@S|@@S|@1\ `$ECHO @S|@2 | $SED -e 's/.*:://' -e 's/:/ /g'`\" eval @S|@1=\"`$ECHO @S|@2 | $SED -e 's/::.*//' -e 's/:/ /g'`\ \@S|@@S|@1\" ;; *) eval @S|@1=\"`$ECHO @S|@2 | $SED 's/:/ /g'`\" ;; esac } ]])# _LT_PREPARE_PATH_LIST # _LT_SYS_DYNAMIC_LINKER([TAG]) # ----------------------------- # PORTME Fill in your ld.so characteristics m4_defun([_LT_SYS_DYNAMIC_LINKER], [AC_REQUIRE([AC_CANONICAL_HOST])dnl m4_require([_LT_DECL_EGREP])dnl m4_require([_LT_FILEUTILS_DEFAULTS])dnl m4_require([_LT_DECL_OBJDUMP])dnl m4_require([_LT_DECL_SED])dnl m4_require([_LT_CHECK_SHELL_FEATURES])dnl m4_require([_LT_PREPARE_MUNGE_PATH_LIST])dnl AC_MSG_CHECKING([dynamic linker characteristics]) m4_if([$1], [], [ if test yes = "$GCC"; then case $host_os in darwin*) lt_awk_arg='/^libraries:/,/LR/' ;; *) lt_awk_arg='/^libraries:/' ;; esac case $host_os in mingw* | cegcc*) lt_sed_strip_eq='s|=\([[A-Za-z]]:\)|\1|g' ;; *) lt_sed_strip_eq='s|=/|/|g' ;; esac lt_search_path_spec=`$CC -print-search-dirs | awk $lt_awk_arg | $SED -e "s/^libraries://" -e $lt_sed_strip_eq` case $lt_search_path_spec in *\;*) # if the path contains ";" then we assume it to be the separator # otherwise default to the standard path separator (i.e. ":") - it is # assumed that no part of a normal pathname contains ";" but that should # okay in the real world where ";" in dirpaths is itself problematic. lt_search_path_spec=`$ECHO "$lt_search_path_spec" | $SED 's/;/ /g'` ;; *) lt_search_path_spec=`$ECHO "$lt_search_path_spec" | $SED "s/$PATH_SEPARATOR/ /g"` ;; esac # Ok, now we have the path, separated by spaces, we can step through it # and add multilib dir if necessary... lt_tmp_lt_search_path_spec= lt_multi_os_dir=/`$CC $CPPFLAGS $CFLAGS $LDFLAGS -print-multi-os-directory 2>/dev/null` # ...but if some path component already ends with the multilib dir we assume # that all is fine and trust -print-search-dirs as is (GCC 4.2? or newer). case "$lt_multi_os_dir; $lt_search_path_spec " in "/; "* | "/.; "* | "/./; "* | *"$lt_multi_os_dir "* | *"$lt_multi_os_dir/ "*) lt_multi_os_dir= ;; esac for lt_sys_path in $lt_search_path_spec; do if test -d "$lt_sys_path$lt_multi_os_dir"; then lt_tmp_lt_search_path_spec="$lt_tmp_lt_search_path_spec $lt_sys_path$lt_multi_os_dir" elif test -n "$lt_multi_os_dir"; then test -d "$lt_sys_path" && \ lt_tmp_lt_search_path_spec="$lt_tmp_lt_search_path_spec $lt_sys_path" fi done lt_search_path_spec=`$ECHO "$lt_tmp_lt_search_path_spec" | awk ' BEGIN {RS = " "; FS = "/|\n";} { lt_foo = ""; lt_count = 0; for (lt_i = NF; lt_i > 0; lt_i--) { if ($lt_i != "" && $lt_i != ".") { if ($lt_i == "..") { lt_count++; } else { if (lt_count == 0) { lt_foo = "/" $lt_i lt_foo; } else { lt_count--; } } } } if (lt_foo != "") { lt_freq[[lt_foo]]++; } if (lt_freq[[lt_foo]] == 1) { print lt_foo; } }'` # AWK program above erroneously prepends '/' to C:/dos/paths # for these hosts. case $host_os in mingw* | cegcc*) lt_search_path_spec=`$ECHO "$lt_search_path_spec" |\ $SED 's|/\([[A-Za-z]]:\)|\1|g'` ;; esac sys_lib_search_path_spec=`$ECHO "$lt_search_path_spec" | $lt_NL2SP` else sys_lib_search_path_spec="/lib /usr/lib /usr/local/lib" fi]) library_names_spec= libname_spec='lib$name' soname_spec= shrext_cmds=.so postinstall_cmds= postuninstall_cmds= finish_cmds= finish_eval= shlibpath_var= shlibpath_overrides_runpath=unknown version_type=none dynamic_linker="$host_os ld.so" sys_lib_dlsearch_path_spec="/lib /usr/lib" need_lib_prefix=unknown hardcode_into_libs=no # when you set need_version to no, make sure it does not cause -set_version # flags to be left without arguments need_version=unknown AC_ARG_VAR([LT_SYS_LIBRARY_PATH], [User-defined run-time library search path.]) case $host_os in aix3*) version_type=linux # correct to gnu/linux during the next big refactor library_names_spec='$libname$release$shared_ext$versuffix $libname.a' shlibpath_var=LIBPATH # AIX 3 has no versioning support, so we append a major version to the name. soname_spec='$libname$release$shared_ext$major' ;; aix[[4-9]]*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no hardcode_into_libs=yes if test ia64 = "$host_cpu"; then # AIX 5 supports IA64 library_names_spec='$libname$release$shared_ext$major $libname$release$shared_ext$versuffix $libname$shared_ext' shlibpath_var=LD_LIBRARY_PATH else # With GCC up to 2.95.x, collect2 would create an import file # for dependence libraries. The import file would start with # the line '#! .'. This would cause the generated library to # depend on '.', always an invalid library. This was fixed in # development snapshots of GCC prior to 3.0. case $host_os in aix4 | aix4.[[01]] | aix4.[[01]].*) if { echo '#if __GNUC__ > 2 || (__GNUC__ == 2 && __GNUC_MINOR__ >= 97)' echo ' yes ' echo '#endif'; } | $CC -E - | $GREP yes > /dev/null; then : else can_build_shared=no fi ;; esac # Using Import Files as archive members, it is possible to support # filename-based versioning of shared library archives on AIX. While # this would work for both with and without runtime linking, it will # prevent static linking of such archives. So we do filename-based # shared library versioning with .so extension only, which is used # when both runtime linking and shared linking is enabled. # Unfortunately, runtime linking may impact performance, so we do # not want this to be the default eventually. Also, we use the # versioned .so libs for executables only if there is the -brtl # linker flag in LDFLAGS as well, or --with-aix-soname=svr4 only. # To allow for filename-based versioning support, we need to create # libNAME.so.V as an archive file, containing: # *) an Import File, referring to the versioned filename of the # archive as well as the shared archive member, telling the # bitwidth (32 or 64) of that shared object, and providing the # list of exported symbols of that shared object, eventually # decorated with the 'weak' keyword # *) the shared object with the F_LOADONLY flag set, to really avoid # it being seen by the linker. # At run time we better use the real file rather than another symlink, # but for link time we create the symlink libNAME.so -> libNAME.so.V case $with_aix_soname,$aix_use_runtimelinking in # AIX (on Power*) has no versioning support, so currently we cannot hardcode correct # soname into executable. Probably we can add versioning support to # collect2, so additional links can be useful in future. aix,yes) # traditional libtool dynamic_linker='AIX unversionable lib.so' # If using run time linking (on AIX 4.2 or later) use lib.so # instead of lib.a to let people know that these are not # typical AIX shared libraries. library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' ;; aix,no) # traditional AIX only dynamic_linker='AIX lib.a[(]lib.so.V[)]' # We preserve .a as extension for shared libraries through AIX4.2 # and later when we are not doing run time linking. library_names_spec='$libname$release.a $libname.a' soname_spec='$libname$release$shared_ext$major' ;; svr4,*) # full svr4 only dynamic_linker="AIX lib.so.V[(]$shared_archive_member_spec.o[)]" library_names_spec='$libname$release$shared_ext$major $libname$shared_ext' # We do not specify a path in Import Files, so LIBPATH fires. shlibpath_overrides_runpath=yes ;; *,yes) # both, prefer svr4 dynamic_linker="AIX lib.so.V[(]$shared_archive_member_spec.o[)], lib.a[(]lib.so.V[)]" library_names_spec='$libname$release$shared_ext$major $libname$shared_ext' # unpreferred sharedlib libNAME.a needs extra handling postinstall_cmds='test -n "$linkname" || linkname="$realname"~func_stripname "" ".so" "$linkname"~$install_shared_prog "$dir/$func_stripname_result.$libext" "$destdir/$func_stripname_result.$libext"~test -z "$tstripme" || test -z "$striplib" || $striplib "$destdir/$func_stripname_result.$libext"' postuninstall_cmds='for n in $library_names $old_library; do :; done~func_stripname "" ".so" "$n"~test "$func_stripname_result" = "$n" || func_append rmfiles " $odir/$func_stripname_result.$libext"' # We do not specify a path in Import Files, so LIBPATH fires. shlibpath_overrides_runpath=yes ;; *,no) # both, prefer aix dynamic_linker="AIX lib.a[(]lib.so.V[)], lib.so.V[(]$shared_archive_member_spec.o[)]" library_names_spec='$libname$release.a $libname.a' soname_spec='$libname$release$shared_ext$major' # unpreferred sharedlib libNAME.so.V and symlink libNAME.so need extra handling postinstall_cmds='test -z "$dlname" || $install_shared_prog $dir/$dlname $destdir/$dlname~test -z "$tstripme" || test -z "$striplib" || $striplib $destdir/$dlname~test -n "$linkname" || linkname=$realname~func_stripname "" ".a" "$linkname"~(cd "$destdir" && $LN_S -f $dlname $func_stripname_result.so)' postuninstall_cmds='test -z "$dlname" || func_append rmfiles " $odir/$dlname"~for n in $old_library $library_names; do :; done~func_stripname "" ".a" "$n"~func_append rmfiles " $odir/$func_stripname_result.so"' ;; esac shlibpath_var=LIBPATH fi ;; amigaos*) case $host_cpu in powerpc) # Since July 2007 AmigaOS4 officially supports .so libraries. # When compiling the executable, add -use-dynld -Lsobjs: to the compileline. library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' ;; m68k) library_names_spec='$libname.ixlibrary $libname.a' # Create ${libname}_ixlibrary.a entries in /sys/libs. finish_eval='for lib in `ls $libdir/*.ixlibrary 2>/dev/null`; do libname=`func_echo_all "$lib" | $SED '\''s%^.*/\([[^/]]*\)\.ixlibrary$%\1%'\''`; $RM /sys/libs/${libname}_ixlibrary.a; $show "cd /sys/libs && $LN_S $lib ${libname}_ixlibrary.a"; cd /sys/libs && $LN_S $lib ${libname}_ixlibrary.a || exit 1; done' ;; esac ;; beos*) library_names_spec='$libname$shared_ext' dynamic_linker="$host_os ld.so" shlibpath_var=LIBRARY_PATH ;; bsdi[[45]]*) version_type=linux # correct to gnu/linux during the next big refactor need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' finish_cmds='PATH="\$PATH:/sbin" ldconfig $libdir' shlibpath_var=LD_LIBRARY_PATH sys_lib_search_path_spec="/shlib /usr/lib /usr/X11/lib /usr/contrib/lib /lib /usr/local/lib" sys_lib_dlsearch_path_spec="/shlib /usr/lib /usr/local/lib" # the default ld.so.conf also contains /usr/contrib/lib and # /usr/X11R6/lib (/usr/X11 is a link to /usr/X11R6), but let us allow # libtool to hard-code these into programs ;; cygwin* | mingw* | pw32* | cegcc*) version_type=windows shrext_cmds=.dll need_version=no need_lib_prefix=no case $GCC,$cc_basename in yes,*) # gcc library_names_spec='$libname.dll.a' # DLL is installed to $(libdir)/../bin by postinstall_cmds postinstall_cmds='base_file=`basename \$file`~ dlpath=`$SHELL 2>&1 -c '\''. $dir/'\''\$base_file'\''i; echo \$dlname'\''`~ dldir=$destdir/`dirname \$dlpath`~ test -d \$dldir || mkdir -p \$dldir~ $install_prog $dir/$dlname \$dldir/$dlname~ chmod a+x \$dldir/$dlname~ if test -n '\''$stripme'\'' && test -n '\''$striplib'\''; then eval '\''$striplib \$dldir/$dlname'\'' || exit \$?; fi' postuninstall_cmds='dldll=`$SHELL 2>&1 -c '\''. $file; echo \$dlname'\''`~ dlpath=$dir/\$dldll~ $RM \$dlpath' shlibpath_overrides_runpath=yes case $host_os in cygwin*) # Cygwin DLLs use 'cyg' prefix rather than 'lib' soname_spec='`echo $libname | sed -e 's/^lib/cyg/'``echo $release | $SED -e 's/[[.]]/-/g'`$versuffix$shared_ext' m4_if([$1], [],[ sys_lib_search_path_spec="$sys_lib_search_path_spec /usr/lib/w32api"]) ;; mingw* | cegcc*) # MinGW DLLs use traditional 'lib' prefix soname_spec='$libname`echo $release | $SED -e 's/[[.]]/-/g'`$versuffix$shared_ext' ;; pw32*) # pw32 DLLs use 'pw' prefix rather than 'lib' library_names_spec='`echo $libname | sed -e 's/^lib/pw/'``echo $release | $SED -e 's/[[.]]/-/g'`$versuffix$shared_ext' ;; esac dynamic_linker='Win32 ld.exe' ;; *,cl*) # Native MSVC libname_spec='$name' soname_spec='$libname`echo $release | $SED -e 's/[[.]]/-/g'`$versuffix$shared_ext' library_names_spec='$libname.dll.lib' case $build_os in mingw*) sys_lib_search_path_spec= lt_save_ifs=$IFS IFS=';' for lt_path in $LIB do IFS=$lt_save_ifs # Let DOS variable expansion print the short 8.3 style file name. lt_path=`cd "$lt_path" 2>/dev/null && cmd //C "for %i in (".") do @echo %~si"` sys_lib_search_path_spec="$sys_lib_search_path_spec $lt_path" done IFS=$lt_save_ifs # Convert to MSYS style. sys_lib_search_path_spec=`$ECHO "$sys_lib_search_path_spec" | sed -e 's|\\\\|/|g' -e 's| \\([[a-zA-Z]]\\):| /\\1|g' -e 's|^ ||'` ;; cygwin*) # Convert to unix form, then to dos form, then back to unix form # but this time dos style (no spaces!) so that the unix form looks # like /cygdrive/c/PROGRA~1:/cygdr... sys_lib_search_path_spec=`cygpath --path --unix "$LIB"` sys_lib_search_path_spec=`cygpath --path --dos "$sys_lib_search_path_spec" 2>/dev/null` sys_lib_search_path_spec=`cygpath --path --unix "$sys_lib_search_path_spec" | $SED -e "s/$PATH_SEPARATOR/ /g"` ;; *) sys_lib_search_path_spec=$LIB if $ECHO "$sys_lib_search_path_spec" | [$GREP ';[c-zC-Z]:/' >/dev/null]; then # It is most probably a Windows format PATH. sys_lib_search_path_spec=`$ECHO "$sys_lib_search_path_spec" | $SED -e 's/;/ /g'` else sys_lib_search_path_spec=`$ECHO "$sys_lib_search_path_spec" | $SED -e "s/$PATH_SEPARATOR/ /g"` fi # FIXME: find the short name or the path components, as spaces are # common. (e.g. "Program Files" -> "PROGRA~1") ;; esac # DLL is installed to $(libdir)/../bin by postinstall_cmds postinstall_cmds='base_file=`basename \$file`~ dlpath=`$SHELL 2>&1 -c '\''. $dir/'\''\$base_file'\''i; echo \$dlname'\''`~ dldir=$destdir/`dirname \$dlpath`~ test -d \$dldir || mkdir -p \$dldir~ $install_prog $dir/$dlname \$dldir/$dlname' postuninstall_cmds='dldll=`$SHELL 2>&1 -c '\''. $file; echo \$dlname'\''`~ dlpath=$dir/\$dldll~ $RM \$dlpath' shlibpath_overrides_runpath=yes dynamic_linker='Win32 link.exe' ;; *) # Assume MSVC wrapper library_names_spec='$libname`echo $release | $SED -e 's/[[.]]/-/g'`$versuffix$shared_ext $libname.lib' dynamic_linker='Win32 ld.exe' ;; esac # FIXME: first we should search . and the directory the executable is in shlibpath_var=PATH ;; darwin* | rhapsody*) dynamic_linker="$host_os dyld" version_type=darwin need_lib_prefix=no need_version=no library_names_spec='$libname$release$major$shared_ext $libname$shared_ext' soname_spec='$libname$release$major$shared_ext' shlibpath_overrides_runpath=yes shlibpath_var=DYLD_LIBRARY_PATH shrext_cmds='`test .$module = .yes && echo .so || echo .dylib`' m4_if([$1], [],[ sys_lib_search_path_spec="$sys_lib_search_path_spec /usr/local/lib"]) sys_lib_dlsearch_path_spec='/usr/local/lib /lib /usr/lib' ;; dgux*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' shlibpath_var=LD_LIBRARY_PATH ;; freebsd* | dragonfly*) # DragonFly does not have aout. When/if they implement a new # versioning mechanism, adjust this. if test -x /usr/bin/objformat; then objformat=`/usr/bin/objformat` else case $host_os in freebsd[[23]].*) objformat=aout ;; *) objformat=elf ;; esac fi version_type=freebsd-$objformat case $version_type in freebsd-elf*) library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' need_version=no need_lib_prefix=no ;; freebsd-*) library_names_spec='$libname$release$shared_ext$versuffix $libname$shared_ext$versuffix' need_version=yes ;; esac shlibpath_var=LD_LIBRARY_PATH case $host_os in freebsd2.*) shlibpath_overrides_runpath=yes ;; freebsd3.[[01]]* | freebsdelf3.[[01]]*) shlibpath_overrides_runpath=yes hardcode_into_libs=yes ;; freebsd3.[[2-9]]* | freebsdelf3.[[2-9]]* | \ freebsd4.[[0-5]] | freebsdelf4.[[0-5]] | freebsd4.1.1 | freebsdelf4.1.1) shlibpath_overrides_runpath=no hardcode_into_libs=yes ;; *) # from 4.6 on, and DragonFly shlibpath_overrides_runpath=yes hardcode_into_libs=yes ;; esac ;; haiku*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no dynamic_linker="$host_os runtime_loader" library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' shlibpath_var=LIBRARY_PATH shlibpath_overrides_runpath=no sys_lib_dlsearch_path_spec='/boot/home/config/lib /boot/common/lib /boot/system/lib' hardcode_into_libs=yes ;; hpux9* | hpux10* | hpux11*) # Give a soname corresponding to the major version so that dld.sl refuses to # link against other versions. version_type=sunos need_lib_prefix=no need_version=no case $host_cpu in ia64*) shrext_cmds='.so' hardcode_into_libs=yes dynamic_linker="$host_os dld.so" shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes # Unless +noenvvar is specified. library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' if test 32 = "$HPUX_IA64_MODE"; then sys_lib_search_path_spec="/usr/lib/hpux32 /usr/local/lib/hpux32 /usr/local/lib" sys_lib_dlsearch_path_spec=/usr/lib/hpux32 else sys_lib_search_path_spec="/usr/lib/hpux64 /usr/local/lib/hpux64" sys_lib_dlsearch_path_spec=/usr/lib/hpux64 fi ;; hppa*64*) shrext_cmds='.sl' hardcode_into_libs=yes dynamic_linker="$host_os dld.sl" shlibpath_var=LD_LIBRARY_PATH # How should we handle SHLIB_PATH shlibpath_overrides_runpath=yes # Unless +noenvvar is specified. library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' sys_lib_search_path_spec="/usr/lib/pa20_64 /usr/ccs/lib/pa20_64" sys_lib_dlsearch_path_spec=$sys_lib_search_path_spec ;; *) shrext_cmds='.sl' dynamic_linker="$host_os dld.sl" shlibpath_var=SHLIB_PATH shlibpath_overrides_runpath=no # +s is required to enable SHLIB_PATH library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' ;; esac # HP-UX runs *really* slowly unless shared libraries are mode 555, ... postinstall_cmds='chmod 555 $lib' # or fails outright, so override atomically: install_override_mode=555 ;; interix[[3-9]]*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' dynamic_linker='Interix 3.x ld.so.1 (PE, like ELF)' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=no hardcode_into_libs=yes ;; irix5* | irix6* | nonstopux*) case $host_os in nonstopux*) version_type=nonstopux ;; *) if test yes = "$lt_cv_prog_gnu_ld"; then version_type=linux # correct to gnu/linux during the next big refactor else version_type=irix fi ;; esac need_lib_prefix=no need_version=no soname_spec='$libname$release$shared_ext$major' library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$release$shared_ext $libname$shared_ext' case $host_os in irix5* | nonstopux*) libsuff= shlibsuff= ;; *) case $LD in # libtool.m4 will add one of these switches to LD *-32|*"-32 "|*-melf32bsmip|*"-melf32bsmip ") libsuff= shlibsuff= libmagic=32-bit;; *-n32|*"-n32 "|*-melf32bmipn32|*"-melf32bmipn32 ") libsuff=32 shlibsuff=N32 libmagic=N32;; *-64|*"-64 "|*-melf64bmip|*"-melf64bmip ") libsuff=64 shlibsuff=64 libmagic=64-bit;; *) libsuff= shlibsuff= libmagic=never-match;; esac ;; esac shlibpath_var=LD_LIBRARY${shlibsuff}_PATH shlibpath_overrides_runpath=no sys_lib_search_path_spec="/usr/lib$libsuff /lib$libsuff /usr/local/lib$libsuff" sys_lib_dlsearch_path_spec="/usr/lib$libsuff /lib$libsuff" hardcode_into_libs=yes ;; # No shared lib support for Linux oldld, aout, or coff. linux*oldld* | linux*aout* | linux*coff*) dynamic_linker=no ;; linux*android*) version_type=none # Android doesn't support versioned libraries. need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext' soname_spec='$libname$release$shared_ext' finish_cmds= shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes # This implies no fast_install, which is unacceptable. # Some rework will be needed to allow for fast_install # before this can be enabled. hardcode_into_libs=yes dynamic_linker='Android linker' # Don't embed -rpath directories since the linker doesn't support them. _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' ;; # This must be glibc/ELF. linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' finish_cmds='PATH="\$PATH:/sbin" ldconfig -n $libdir' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=no # Some binutils ld are patched to set DT_RUNPATH AC_CACHE_VAL([lt_cv_shlibpath_overrides_runpath], [lt_cv_shlibpath_overrides_runpath=no save_LDFLAGS=$LDFLAGS save_libdir=$libdir eval "libdir=/foo; wl=\"$_LT_TAGVAR(lt_prog_compiler_wl, $1)\"; \ LDFLAGS=\"\$LDFLAGS $_LT_TAGVAR(hardcode_libdir_flag_spec, $1)\"" AC_LINK_IFELSE([AC_LANG_PROGRAM([],[])], [AS_IF([ ($OBJDUMP -p conftest$ac_exeext) 2>/dev/null | grep "RUNPATH.*$libdir" >/dev/null], [lt_cv_shlibpath_overrides_runpath=yes])]) LDFLAGS=$save_LDFLAGS libdir=$save_libdir ]) shlibpath_overrides_runpath=$lt_cv_shlibpath_overrides_runpath # This implies no fast_install, which is unacceptable. # Some rework will be needed to allow for fast_install # before this can be enabled. hardcode_into_libs=yes # Ideally, we could use ldconfig to report *all* directores which are # searched for libraries, however this is still not possible. Aside from not # being certain /sbin/ldconfig is available, command # 'ldconfig -N -X -v | grep ^/' on 64bit Fedora does not report /usr/lib64, # even though it is searched at run-time. Try to do the best guess by # appending ld.so.conf contents (and includes) to the search path. if test -f /etc/ld.so.conf; then lt_ld_extra=`awk '/^include / { system(sprintf("cd /etc; cat %s 2>/dev/null", \[$]2)); skip = 1; } { if (!skip) print \[$]0; skip = 0; }' < /etc/ld.so.conf | $SED -e 's/#.*//;/^[ ]*hwcap[ ]/d;s/[:, ]/ /g;s/=[^=]*$//;s/=[^= ]* / /g;s/"//g;/^$/d' | tr '\n' ' '` sys_lib_dlsearch_path_spec="/lib /usr/lib $lt_ld_extra" fi # We used to test for /lib/ld.so.1 and disable shared libraries on # powerpc, because MkLinux only supported shared libraries with the # GNU dynamic linker. Since this was broken with cross compilers, # most powerpc-linux boxes support dynamic linking these days and # people can always --disable-shared, the test was removed, and we # assume the GNU/Linux dynamic linker is in use. dynamic_linker='GNU/Linux ld.so' ;; netbsdelf*-gnu) version_type=linux need_lib_prefix=no need_version=no library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major ${libname}${shared_ext}' soname_spec='${libname}${release}${shared_ext}$major' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=no hardcode_into_libs=yes dynamic_linker='NetBSD ld.elf_so' ;; netbsd*) version_type=sunos need_lib_prefix=no need_version=no if echo __ELF__ | $CC -E - | $GREP __ELF__ >/dev/null; then library_names_spec='$libname$release$shared_ext$versuffix $libname$shared_ext$versuffix' finish_cmds='PATH="\$PATH:/sbin" ldconfig -m $libdir' dynamic_linker='NetBSD (a.out) ld.so' else library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' dynamic_linker='NetBSD ld.elf_so' fi shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes hardcode_into_libs=yes ;; newsos6) version_type=linux # correct to gnu/linux during the next big refactor library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes ;; *nto* | *qnx*) version_type=qnx need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=no hardcode_into_libs=yes dynamic_linker='ldqnx.so' ;; openbsd* | bitrig*) version_type=sunos sys_lib_dlsearch_path_spec=/usr/lib need_lib_prefix=no if test -z "`echo __ELF__ | $CC -E - | $GREP __ELF__`"; then need_version=no else need_version=yes fi library_names_spec='$libname$release$shared_ext$versuffix $libname$shared_ext$versuffix' finish_cmds='PATH="\$PATH:/sbin" ldconfig -m $libdir' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes ;; os2*) libname_spec='$name' version_type=windows shrext_cmds=.dll need_version=no need_lib_prefix=no # OS/2 can only load a DLL with a base name of 8 characters or less. soname_spec='`test -n "$os2dllname" && libname="$os2dllname"; v=$($ECHO $release$versuffix | tr -d .-); n=$($ECHO $libname | cut -b -$((8 - ${#v})) | tr . _); $ECHO $n$v`$shared_ext' library_names_spec='${libname}_dll.$libext' dynamic_linker='OS/2 ld.exe' shlibpath_var=BEGINLIBPATH sys_lib_search_path_spec="/lib /usr/lib /usr/local/lib" sys_lib_dlsearch_path_spec=$sys_lib_search_path_spec postinstall_cmds='base_file=`basename \$file`~ dlpath=`$SHELL 2>&1 -c '\''. $dir/'\''\$base_file'\''i; $ECHO \$dlname'\''`~ dldir=$destdir/`dirname \$dlpath`~ test -d \$dldir || mkdir -p \$dldir~ $install_prog $dir/$dlname \$dldir/$dlname~ chmod a+x \$dldir/$dlname~ if test -n '\''$stripme'\'' && test -n '\''$striplib'\''; then eval '\''$striplib \$dldir/$dlname'\'' || exit \$?; fi' postuninstall_cmds='dldll=`$SHELL 2>&1 -c '\''. $file; $ECHO \$dlname'\''`~ dlpath=$dir/\$dldll~ $RM \$dlpath' ;; osf3* | osf4* | osf5*) version_type=osf need_lib_prefix=no need_version=no soname_spec='$libname$release$shared_ext$major' library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' shlibpath_var=LD_LIBRARY_PATH sys_lib_search_path_spec="/usr/shlib /usr/ccs/lib /usr/lib/cmplrs/cc /usr/lib /usr/local/lib /var/shlib" sys_lib_dlsearch_path_spec=$sys_lib_search_path_spec ;; rdos*) dynamic_linker=no ;; solaris*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes hardcode_into_libs=yes # ldd complains unless libraries are executable postinstall_cmds='chmod +x $lib' ;; sunos4*) version_type=sunos library_names_spec='$libname$release$shared_ext$versuffix $libname$shared_ext$versuffix' finish_cmds='PATH="\$PATH:/usr/etc" ldconfig $libdir' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes if test yes = "$with_gnu_ld"; then need_lib_prefix=no fi need_version=yes ;; sysv4 | sysv4.3*) version_type=linux # correct to gnu/linux during the next big refactor library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' shlibpath_var=LD_LIBRARY_PATH case $host_vendor in sni) shlibpath_overrides_runpath=no need_lib_prefix=no runpath_var=LD_RUN_PATH ;; siemens) need_lib_prefix=no ;; motorola) need_lib_prefix=no need_version=no shlibpath_overrides_runpath=no sys_lib_search_path_spec='/lib /usr/lib /usr/ccs/lib' ;; esac ;; sysv4*MP*) if test -d /usr/nec; then version_type=linux # correct to gnu/linux during the next big refactor library_names_spec='$libname$shared_ext.$versuffix $libname$shared_ext.$major $libname$shared_ext' soname_spec='$libname$shared_ext.$major' shlibpath_var=LD_LIBRARY_PATH fi ;; sysv5* | sco3.2v5* | sco5v6* | unixware* | OpenUNIX* | sysv4*uw2*) version_type=sco need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes hardcode_into_libs=yes if test yes = "$with_gnu_ld"; then sys_lib_search_path_spec='/usr/local/lib /usr/gnu/lib /usr/ccs/lib /usr/lib /lib' else sys_lib_search_path_spec='/usr/ccs/lib /usr/lib' case $host_os in sco3.2v5*) sys_lib_search_path_spec="$sys_lib_search_path_spec /lib" ;; esac fi sys_lib_dlsearch_path_spec='/usr/lib' ;; tpf*) # TPF is a cross-target only. Preferred cross-host = GNU/Linux. version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=no hardcode_into_libs=yes ;; uts4*) version_type=linux # correct to gnu/linux during the next big refactor library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' shlibpath_var=LD_LIBRARY_PATH ;; *) dynamic_linker=no ;; esac AC_MSG_RESULT([$dynamic_linker]) test no = "$dynamic_linker" && can_build_shared=no variables_saved_for_relink="PATH $shlibpath_var $runpath_var" if test yes = "$GCC"; then variables_saved_for_relink="$variables_saved_for_relink GCC_EXEC_PREFIX COMPILER_PATH LIBRARY_PATH" fi if test set = "${lt_cv_sys_lib_search_path_spec+set}"; then sys_lib_search_path_spec=$lt_cv_sys_lib_search_path_spec fi if test set = "${lt_cv_sys_lib_dlsearch_path_spec+set}"; then sys_lib_dlsearch_path_spec=$lt_cv_sys_lib_dlsearch_path_spec fi # remember unaugmented sys_lib_dlsearch_path content for libtool script decls... configure_time_dlsearch_path=$sys_lib_dlsearch_path_spec # ... but it needs LT_SYS_LIBRARY_PATH munging for other configure-time code func_munge_path_list sys_lib_dlsearch_path_spec "$LT_SYS_LIBRARY_PATH" # to be used as default LT_SYS_LIBRARY_PATH value in generated libtool configure_time_lt_sys_library_path=$LT_SYS_LIBRARY_PATH _LT_DECL([], [variables_saved_for_relink], [1], [Variables whose values should be saved in libtool wrapper scripts and restored at link time]) _LT_DECL([], [need_lib_prefix], [0], [Do we need the "lib" prefix for modules?]) _LT_DECL([], [need_version], [0], [Do we need a version for libraries?]) _LT_DECL([], [version_type], [0], [Library versioning type]) _LT_DECL([], [runpath_var], [0], [Shared library runtime path variable]) _LT_DECL([], [shlibpath_var], [0],[Shared library path variable]) _LT_DECL([], [shlibpath_overrides_runpath], [0], [Is shlibpath searched before the hard-coded library search path?]) _LT_DECL([], [libname_spec], [1], [Format of library name prefix]) _LT_DECL([], [library_names_spec], [1], [[List of archive names. First name is the real one, the rest are links. The last name is the one that the linker finds with -lNAME]]) _LT_DECL([], [soname_spec], [1], [[The coded name of the library, if different from the real name]]) _LT_DECL([], [install_override_mode], [1], [Permission mode override for installation of shared libraries]) _LT_DECL([], [postinstall_cmds], [2], [Command to use after installation of a shared archive]) _LT_DECL([], [postuninstall_cmds], [2], [Command to use after uninstallation of a shared archive]) _LT_DECL([], [finish_cmds], [2], [Commands used to finish a libtool library installation in a directory]) _LT_DECL([], [finish_eval], [1], [[As "finish_cmds", except a single script fragment to be evaled but not shown]]) _LT_DECL([], [hardcode_into_libs], [0], [Whether we should hardcode library paths into libraries]) _LT_DECL([], [sys_lib_search_path_spec], [2], [Compile-time system search path for libraries]) _LT_DECL([sys_lib_dlsearch_path_spec], [configure_time_dlsearch_path], [2], [Detected run-time system search path for libraries]) _LT_DECL([], [configure_time_lt_sys_library_path], [2], [Explicit LT_SYS_LIBRARY_PATH set during ./configure time]) ])# _LT_SYS_DYNAMIC_LINKER # _LT_PATH_TOOL_PREFIX(TOOL) # -------------------------- # find a file program that can recognize shared library AC_DEFUN([_LT_PATH_TOOL_PREFIX], [m4_require([_LT_DECL_EGREP])dnl AC_MSG_CHECKING([for $1]) AC_CACHE_VAL(lt_cv_path_MAGIC_CMD, [case $MAGIC_CMD in [[\\/*] | ?:[\\/]*]) lt_cv_path_MAGIC_CMD=$MAGIC_CMD # Let the user override the test with a path. ;; *) lt_save_MAGIC_CMD=$MAGIC_CMD lt_save_ifs=$IFS; IFS=$PATH_SEPARATOR dnl $ac_dummy forces splitting on constant user-supplied paths. dnl POSIX.2 word splitting is done only on the output of word expansions, dnl not every word. This closes a longstanding sh security hole. ac_dummy="m4_if([$2], , $PATH, [$2])" for ac_dir in $ac_dummy; do IFS=$lt_save_ifs test -z "$ac_dir" && ac_dir=. if test -f "$ac_dir/$1"; then lt_cv_path_MAGIC_CMD=$ac_dir/"$1" if test -n "$file_magic_test_file"; then case $deplibs_check_method in "file_magic "*) file_magic_regex=`expr "$deplibs_check_method" : "file_magic \(.*\)"` MAGIC_CMD=$lt_cv_path_MAGIC_CMD if eval $file_magic_cmd \$file_magic_test_file 2> /dev/null | $EGREP "$file_magic_regex" > /dev/null; then : else cat <<_LT_EOF 1>&2 *** Warning: the command libtool uses to detect shared libraries, *** $file_magic_cmd, produces output that libtool cannot recognize. *** The result is that libtool may fail to recognize shared libraries *** as such. This will affect the creation of libtool libraries that *** depend on shared libraries, but programs linked with such libtool *** libraries will work regardless of this problem. Nevertheless, you *** may want to report the problem to your system manager and/or to *** bug-libtool@gnu.org _LT_EOF fi ;; esac fi break fi done IFS=$lt_save_ifs MAGIC_CMD=$lt_save_MAGIC_CMD ;; esac]) MAGIC_CMD=$lt_cv_path_MAGIC_CMD if test -n "$MAGIC_CMD"; then AC_MSG_RESULT($MAGIC_CMD) else AC_MSG_RESULT(no) fi _LT_DECL([], [MAGIC_CMD], [0], [Used to examine libraries when file_magic_cmd begins with "file"])dnl ])# _LT_PATH_TOOL_PREFIX # Old name: AU_ALIAS([AC_PATH_TOOL_PREFIX], [_LT_PATH_TOOL_PREFIX]) dnl aclocal-1.4 backwards compatibility: dnl AC_DEFUN([AC_PATH_TOOL_PREFIX], []) # _LT_PATH_MAGIC # -------------- # find a file program that can recognize a shared library m4_defun([_LT_PATH_MAGIC], [_LT_PATH_TOOL_PREFIX(${ac_tool_prefix}file, /usr/bin$PATH_SEPARATOR$PATH) if test -z "$lt_cv_path_MAGIC_CMD"; then if test -n "$ac_tool_prefix"; then _LT_PATH_TOOL_PREFIX(file, /usr/bin$PATH_SEPARATOR$PATH) else MAGIC_CMD=: fi fi ])# _LT_PATH_MAGIC # LT_PATH_LD # ---------- # find the pathname to the GNU or non-GNU linker AC_DEFUN([LT_PATH_LD], [AC_REQUIRE([AC_PROG_CC])dnl AC_REQUIRE([AC_CANONICAL_HOST])dnl AC_REQUIRE([AC_CANONICAL_BUILD])dnl m4_require([_LT_DECL_SED])dnl m4_require([_LT_DECL_EGREP])dnl m4_require([_LT_PROG_ECHO_BACKSLASH])dnl AC_ARG_WITH([gnu-ld], [AS_HELP_STRING([--with-gnu-ld], [assume the C compiler uses GNU ld @<:@default=no@:>@])], [test no = "$withval" || with_gnu_ld=yes], [with_gnu_ld=no])dnl ac_prog=ld if test yes = "$GCC"; then # Check if gcc -print-prog-name=ld gives a path. AC_MSG_CHECKING([for ld used by $CC]) case $host in *-*-mingw*) # gcc leaves a trailing carriage return, which upsets mingw ac_prog=`($CC -print-prog-name=ld) 2>&5 | tr -d '\015'` ;; *) ac_prog=`($CC -print-prog-name=ld) 2>&5` ;; esac case $ac_prog in # Accept absolute paths. [[\\/]]* | ?:[[\\/]]*) re_direlt='/[[^/]][[^/]]*/\.\./' # Canonicalize the pathname of ld ac_prog=`$ECHO "$ac_prog"| $SED 's%\\\\%/%g'` while $ECHO "$ac_prog" | $GREP "$re_direlt" > /dev/null 2>&1; do ac_prog=`$ECHO $ac_prog| $SED "s%$re_direlt%/%"` done test -z "$LD" && LD=$ac_prog ;; "") # If it fails, then pretend we aren't using GCC. ac_prog=ld ;; *) # If it is relative, then search for the first ld in PATH. with_gnu_ld=unknown ;; esac elif test yes = "$with_gnu_ld"; then AC_MSG_CHECKING([for GNU ld]) else AC_MSG_CHECKING([for non-GNU ld]) fi AC_CACHE_VAL(lt_cv_path_LD, [if test -z "$LD"; then lt_save_ifs=$IFS; IFS=$PATH_SEPARATOR for ac_dir in $PATH; do IFS=$lt_save_ifs test -z "$ac_dir" && ac_dir=. if test -f "$ac_dir/$ac_prog" || test -f "$ac_dir/$ac_prog$ac_exeext"; then lt_cv_path_LD=$ac_dir/$ac_prog # Check to see if the program is GNU ld. I'd rather use --version, # but apparently some variants of GNU ld only accept -v. # Break only if it was the GNU/non-GNU ld that we prefer. case `"$lt_cv_path_LD" -v 2>&1 &1 conftest.i cat conftest.i conftest.i >conftest2.i : ${lt_DD:=$DD} AC_PATH_PROGS_FEATURE_CHECK([lt_DD], [dd], [if "$ac_path_lt_DD" bs=32 count=1 conftest.out 2>/dev/null; then cmp -s conftest.i conftest.out \ && ac_cv_path_lt_DD="$ac_path_lt_DD" ac_path_lt_DD_found=: fi]) rm -f conftest.i conftest2.i conftest.out]) ])# _LT_PATH_DD # _LT_CMD_TRUNCATE # ---------------- # find command to truncate a binary pipe m4_defun([_LT_CMD_TRUNCATE], [m4_require([_LT_PATH_DD]) AC_CACHE_CHECK([how to truncate binary pipes], [lt_cv_truncate_bin], [printf 0123456789abcdef0123456789abcdef >conftest.i cat conftest.i conftest.i >conftest2.i lt_cv_truncate_bin= if "$ac_cv_path_lt_DD" bs=32 count=1 conftest.out 2>/dev/null; then cmp -s conftest.i conftest.out \ && lt_cv_truncate_bin="$ac_cv_path_lt_DD bs=4096 count=1" fi rm -f conftest.i conftest2.i conftest.out test -z "$lt_cv_truncate_bin" && lt_cv_truncate_bin="$SED -e 4q"]) _LT_DECL([lt_truncate_bin], [lt_cv_truncate_bin], [1], [Command to truncate a binary pipe]) ])# _LT_CMD_TRUNCATE # _LT_CHECK_MAGIC_METHOD # ---------------------- # how to check for library dependencies # -- PORTME fill in with the dynamic library characteristics m4_defun([_LT_CHECK_MAGIC_METHOD], [m4_require([_LT_DECL_EGREP]) m4_require([_LT_DECL_OBJDUMP]) AC_CACHE_CHECK([how to recognize dependent libraries], lt_cv_deplibs_check_method, [lt_cv_file_magic_cmd='$MAGIC_CMD' lt_cv_file_magic_test_file= lt_cv_deplibs_check_method='unknown' # Need to set the preceding variable on all platforms that support # interlibrary dependencies. # 'none' -- dependencies not supported. # 'unknown' -- same as none, but documents that we really don't know. # 'pass_all' -- all dependencies passed with no checks. # 'test_compile' -- check by making test program. # 'file_magic [[regex]]' -- check by looking for files in library path # that responds to the $file_magic_cmd with a given extended regex. # If you have 'file' or equivalent on your system and you're not sure # whether 'pass_all' will *always* work, you probably want this one. case $host_os in aix[[4-9]]*) lt_cv_deplibs_check_method=pass_all ;; beos*) lt_cv_deplibs_check_method=pass_all ;; bsdi[[45]]*) lt_cv_deplibs_check_method='file_magic ELF [[0-9]][[0-9]]*-bit [[ML]]SB (shared object|dynamic lib)' lt_cv_file_magic_cmd='/usr/bin/file -L' lt_cv_file_magic_test_file=/shlib/libc.so ;; cygwin*) # func_win32_libid is a shell function defined in ltmain.sh lt_cv_deplibs_check_method='file_magic ^x86 archive import|^x86 DLL' lt_cv_file_magic_cmd='func_win32_libid' ;; mingw* | pw32*) # Base MSYS/MinGW do not provide the 'file' command needed by # func_win32_libid shell function, so use a weaker test based on 'objdump', # unless we find 'file', for example because we are cross-compiling. if ( file / ) >/dev/null 2>&1; then lt_cv_deplibs_check_method='file_magic ^x86 archive import|^x86 DLL' lt_cv_file_magic_cmd='func_win32_libid' else # Keep this pattern in sync with the one in func_win32_libid. lt_cv_deplibs_check_method='file_magic file format (pei*-i386(.*architecture: i386)?|pe-arm-wince|pe-x86-64)' lt_cv_file_magic_cmd='$OBJDUMP -f' fi ;; cegcc*) # use the weaker test based on 'objdump'. See mingw*. lt_cv_deplibs_check_method='file_magic file format pe-arm-.*little(.*architecture: arm)?' lt_cv_file_magic_cmd='$OBJDUMP -f' ;; darwin* | rhapsody*) lt_cv_deplibs_check_method=pass_all ;; freebsd* | dragonfly*) if echo __ELF__ | $CC -E - | $GREP __ELF__ > /dev/null; then case $host_cpu in i*86 ) # Not sure whether the presence of OpenBSD here was a mistake. # Let's accept both of them until this is cleared up. lt_cv_deplibs_check_method='file_magic (FreeBSD|OpenBSD|DragonFly)/i[[3-9]]86 (compact )?demand paged shared library' lt_cv_file_magic_cmd=/usr/bin/file lt_cv_file_magic_test_file=`echo /usr/lib/libc.so.*` ;; esac else lt_cv_deplibs_check_method=pass_all fi ;; haiku*) lt_cv_deplibs_check_method=pass_all ;; hpux10.20* | hpux11*) lt_cv_file_magic_cmd=/usr/bin/file case $host_cpu in ia64*) lt_cv_deplibs_check_method='file_magic (s[[0-9]][[0-9]][[0-9]]|ELF-[[0-9]][[0-9]]) shared object file - IA64' lt_cv_file_magic_test_file=/usr/lib/hpux32/libc.so ;; hppa*64*) [lt_cv_deplibs_check_method='file_magic (s[0-9][0-9][0-9]|ELF[ -][0-9][0-9])(-bit)?( [LM]SB)? shared object( file)?[, -]* PA-RISC [0-9]\.[0-9]'] lt_cv_file_magic_test_file=/usr/lib/pa20_64/libc.sl ;; *) lt_cv_deplibs_check_method='file_magic (s[[0-9]][[0-9]][[0-9]]|PA-RISC[[0-9]]\.[[0-9]]) shared library' lt_cv_file_magic_test_file=/usr/lib/libc.sl ;; esac ;; interix[[3-9]]*) # PIC code is broken on Interix 3.x, that's why |\.a not |_pic\.a here lt_cv_deplibs_check_method='match_pattern /lib[[^/]]+(\.so|\.a)$' ;; irix5* | irix6* | nonstopux*) case $LD in *-32|*"-32 ") libmagic=32-bit;; *-n32|*"-n32 ") libmagic=N32;; *-64|*"-64 ") libmagic=64-bit;; *) libmagic=never-match;; esac lt_cv_deplibs_check_method=pass_all ;; # This must be glibc/ELF. linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*) lt_cv_deplibs_check_method=pass_all ;; netbsd* | netbsdelf*-gnu) if echo __ELF__ | $CC -E - | $GREP __ELF__ > /dev/null; then lt_cv_deplibs_check_method='match_pattern /lib[[^/]]+(\.so\.[[0-9]]+\.[[0-9]]+|_pic\.a)$' else lt_cv_deplibs_check_method='match_pattern /lib[[^/]]+(\.so|_pic\.a)$' fi ;; newos6*) lt_cv_deplibs_check_method='file_magic ELF [[0-9]][[0-9]]*-bit [[ML]]SB (executable|dynamic lib)' lt_cv_file_magic_cmd=/usr/bin/file lt_cv_file_magic_test_file=/usr/lib/libnls.so ;; *nto* | *qnx*) lt_cv_deplibs_check_method=pass_all ;; openbsd* | bitrig*) if test -z "`echo __ELF__ | $CC -E - | $GREP __ELF__`"; then lt_cv_deplibs_check_method='match_pattern /lib[[^/]]+(\.so\.[[0-9]]+\.[[0-9]]+|\.so|_pic\.a)$' else lt_cv_deplibs_check_method='match_pattern /lib[[^/]]+(\.so\.[[0-9]]+\.[[0-9]]+|_pic\.a)$' fi ;; osf3* | osf4* | osf5*) lt_cv_deplibs_check_method=pass_all ;; rdos*) lt_cv_deplibs_check_method=pass_all ;; solaris*) lt_cv_deplibs_check_method=pass_all ;; sysv5* | sco3.2v5* | sco5v6* | unixware* | OpenUNIX* | sysv4*uw2*) lt_cv_deplibs_check_method=pass_all ;; sysv4 | sysv4.3*) case $host_vendor in motorola) lt_cv_deplibs_check_method='file_magic ELF [[0-9]][[0-9]]*-bit [[ML]]SB (shared object|dynamic lib) M[[0-9]][[0-9]]* Version [[0-9]]' lt_cv_file_magic_test_file=`echo /usr/lib/libc.so*` ;; ncr) lt_cv_deplibs_check_method=pass_all ;; sequent) lt_cv_file_magic_cmd='/bin/file' lt_cv_deplibs_check_method='file_magic ELF [[0-9]][[0-9]]*-bit [[LM]]SB (shared object|dynamic lib )' ;; sni) lt_cv_file_magic_cmd='/bin/file' lt_cv_deplibs_check_method="file_magic ELF [[0-9]][[0-9]]*-bit [[LM]]SB dynamic lib" lt_cv_file_magic_test_file=/lib/libc.so ;; siemens) lt_cv_deplibs_check_method=pass_all ;; pc) lt_cv_deplibs_check_method=pass_all ;; esac ;; tpf*) lt_cv_deplibs_check_method=pass_all ;; os2*) lt_cv_deplibs_check_method=pass_all ;; esac ]) file_magic_glob= want_nocaseglob=no if test "$build" = "$host"; then case $host_os in mingw* | pw32*) if ( shopt | grep nocaseglob ) >/dev/null 2>&1; then want_nocaseglob=yes else file_magic_glob=`echo aAbBcCdDeEfFgGhHiIjJkKlLmMnNoOpPqQrRsStTuUvVwWxXyYzZ | $SED -e "s/\(..\)/s\/[[\1]]\/[[\1]]\/g;/g"` fi ;; esac fi file_magic_cmd=$lt_cv_file_magic_cmd deplibs_check_method=$lt_cv_deplibs_check_method test -z "$deplibs_check_method" && deplibs_check_method=unknown _LT_DECL([], [deplibs_check_method], [1], [Method to check whether dependent libraries are shared objects]) _LT_DECL([], [file_magic_cmd], [1], [Command to use when deplibs_check_method = "file_magic"]) _LT_DECL([], [file_magic_glob], [1], [How to find potential files when deplibs_check_method = "file_magic"]) _LT_DECL([], [want_nocaseglob], [1], [Find potential files using nocaseglob when deplibs_check_method = "file_magic"]) ])# _LT_CHECK_MAGIC_METHOD # LT_PATH_NM # ---------- # find the pathname to a BSD- or MS-compatible name lister AC_DEFUN([LT_PATH_NM], [AC_REQUIRE([AC_PROG_CC])dnl AC_CACHE_CHECK([for BSD- or MS-compatible name lister (nm)], lt_cv_path_NM, [if test -n "$NM"; then # Let the user override the test. lt_cv_path_NM=$NM else lt_nm_to_check=${ac_tool_prefix}nm if test -n "$ac_tool_prefix" && test "$build" = "$host"; then lt_nm_to_check="$lt_nm_to_check nm" fi for lt_tmp_nm in $lt_nm_to_check; do lt_save_ifs=$IFS; IFS=$PATH_SEPARATOR for ac_dir in $PATH /usr/ccs/bin/elf /usr/ccs/bin /usr/ucb /bin; do IFS=$lt_save_ifs test -z "$ac_dir" && ac_dir=. tmp_nm=$ac_dir/$lt_tmp_nm if test -f "$tmp_nm" || test -f "$tmp_nm$ac_exeext"; then # Check to see if the nm accepts a BSD-compat flag. # Adding the 'sed 1q' prevents false positives on HP-UX, which says: # nm: unknown option "B" ignored # Tru64's nm complains that /dev/null is an invalid object file # MSYS converts /dev/null to NUL, MinGW nm treats NUL as empty case $build_os in mingw*) lt_bad_file=conftest.nm/nofile ;; *) lt_bad_file=/dev/null ;; esac case `"$tmp_nm" -B $lt_bad_file 2>&1 | sed '1q'` in *$lt_bad_file* | *'Invalid file or object type'*) lt_cv_path_NM="$tmp_nm -B" break 2 ;; *) case `"$tmp_nm" -p /dev/null 2>&1 | sed '1q'` in */dev/null*) lt_cv_path_NM="$tmp_nm -p" break 2 ;; *) lt_cv_path_NM=${lt_cv_path_NM="$tmp_nm"} # keep the first match, but continue # so that we can try to find one that supports BSD flags ;; esac ;; esac fi done IFS=$lt_save_ifs done : ${lt_cv_path_NM=no} fi]) if test no != "$lt_cv_path_NM"; then NM=$lt_cv_path_NM else # Didn't find any BSD compatible name lister, look for dumpbin. if test -n "$DUMPBIN"; then : # Let the user override the test. else AC_CHECK_TOOLS(DUMPBIN, [dumpbin "link -dump"], :) case `$DUMPBIN -symbols -headers /dev/null 2>&1 | sed '1q'` in *COFF*) DUMPBIN="$DUMPBIN -symbols -headers" ;; *) DUMPBIN=: ;; esac fi AC_SUBST([DUMPBIN]) if test : != "$DUMPBIN"; then NM=$DUMPBIN fi fi test -z "$NM" && NM=nm AC_SUBST([NM]) _LT_DECL([], [NM], [1], [A BSD- or MS-compatible name lister])dnl AC_CACHE_CHECK([the name lister ($NM) interface], [lt_cv_nm_interface], [lt_cv_nm_interface="BSD nm" echo "int some_variable = 0;" > conftest.$ac_ext (eval echo "\"\$as_me:$LINENO: $ac_compile\"" >&AS_MESSAGE_LOG_FD) (eval "$ac_compile" 2>conftest.err) cat conftest.err >&AS_MESSAGE_LOG_FD (eval echo "\"\$as_me:$LINENO: $NM \\\"conftest.$ac_objext\\\"\"" >&AS_MESSAGE_LOG_FD) (eval "$NM \"conftest.$ac_objext\"" 2>conftest.err > conftest.out) cat conftest.err >&AS_MESSAGE_LOG_FD (eval echo "\"\$as_me:$LINENO: output\"" >&AS_MESSAGE_LOG_FD) cat conftest.out >&AS_MESSAGE_LOG_FD if $GREP 'External.*some_variable' conftest.out > /dev/null; then lt_cv_nm_interface="MS dumpbin" fi rm -f conftest*]) ])# LT_PATH_NM # Old names: AU_ALIAS([AM_PROG_NM], [LT_PATH_NM]) AU_ALIAS([AC_PROG_NM], [LT_PATH_NM]) dnl aclocal-1.4 backwards compatibility: dnl AC_DEFUN([AM_PROG_NM], []) dnl AC_DEFUN([AC_PROG_NM], []) # _LT_CHECK_SHAREDLIB_FROM_LINKLIB # -------------------------------- # how to determine the name of the shared library # associated with a specific link library. # -- PORTME fill in with the dynamic library characteristics m4_defun([_LT_CHECK_SHAREDLIB_FROM_LINKLIB], [m4_require([_LT_DECL_EGREP]) m4_require([_LT_DECL_OBJDUMP]) m4_require([_LT_DECL_DLLTOOL]) AC_CACHE_CHECK([how to associate runtime and link libraries], lt_cv_sharedlib_from_linklib_cmd, [lt_cv_sharedlib_from_linklib_cmd='unknown' case $host_os in cygwin* | mingw* | pw32* | cegcc*) # two different shell functions defined in ltmain.sh; # decide which one to use based on capabilities of $DLLTOOL case `$DLLTOOL --help 2>&1` in *--identify-strict*) lt_cv_sharedlib_from_linklib_cmd=func_cygming_dll_for_implib ;; *) lt_cv_sharedlib_from_linklib_cmd=func_cygming_dll_for_implib_fallback ;; esac ;; *) # fallback: assume linklib IS sharedlib lt_cv_sharedlib_from_linklib_cmd=$ECHO ;; esac ]) sharedlib_from_linklib_cmd=$lt_cv_sharedlib_from_linklib_cmd test -z "$sharedlib_from_linklib_cmd" && sharedlib_from_linklib_cmd=$ECHO _LT_DECL([], [sharedlib_from_linklib_cmd], [1], [Command to associate shared and link libraries]) ])# _LT_CHECK_SHAREDLIB_FROM_LINKLIB # _LT_PATH_MANIFEST_TOOL # ---------------------- # locate the manifest tool m4_defun([_LT_PATH_MANIFEST_TOOL], [AC_CHECK_TOOL(MANIFEST_TOOL, mt, :) test -z "$MANIFEST_TOOL" && MANIFEST_TOOL=mt AC_CACHE_CHECK([if $MANIFEST_TOOL is a manifest tool], [lt_cv_path_mainfest_tool], [lt_cv_path_mainfest_tool=no echo "$as_me:$LINENO: $MANIFEST_TOOL '-?'" >&AS_MESSAGE_LOG_FD $MANIFEST_TOOL '-?' 2>conftest.err > conftest.out cat conftest.err >&AS_MESSAGE_LOG_FD if $GREP 'Manifest Tool' conftest.out > /dev/null; then lt_cv_path_mainfest_tool=yes fi rm -f conftest*]) if test yes != "$lt_cv_path_mainfest_tool"; then MANIFEST_TOOL=: fi _LT_DECL([], [MANIFEST_TOOL], [1], [Manifest tool])dnl ])# _LT_PATH_MANIFEST_TOOL # _LT_DLL_DEF_P([FILE]) # --------------------- # True iff FILE is a Windows DLL '.def' file. # Keep in sync with func_dll_def_p in the libtool script AC_DEFUN([_LT_DLL_DEF_P], [dnl test DEF = "`$SED -n dnl -e '\''s/^[[ ]]*//'\'' dnl Strip leading whitespace -e '\''/^\(;.*\)*$/d'\'' dnl Delete empty lines and comments -e '\''s/^\(EXPORTS\|LIBRARY\)\([[ ]].*\)*$/DEF/p'\'' dnl -e q dnl Only consider the first "real" line $1`" dnl ])# _LT_DLL_DEF_P # LT_LIB_M # -------- # check for math library AC_DEFUN([LT_LIB_M], [AC_REQUIRE([AC_CANONICAL_HOST])dnl LIBM= case $host in *-*-beos* | *-*-cegcc* | *-*-cygwin* | *-*-haiku* | *-*-pw32* | *-*-darwin*) # These system don't have libm, or don't need it ;; *-ncr-sysv4.3*) AC_CHECK_LIB(mw, _mwvalidcheckl, LIBM=-lmw) AC_CHECK_LIB(m, cos, LIBM="$LIBM -lm") ;; *) AC_CHECK_LIB(m, cos, LIBM=-lm) ;; esac AC_SUBST([LIBM]) ])# LT_LIB_M # Old name: AU_ALIAS([AC_CHECK_LIBM], [LT_LIB_M]) dnl aclocal-1.4 backwards compatibility: dnl AC_DEFUN([AC_CHECK_LIBM], []) # _LT_COMPILER_NO_RTTI([TAGNAME]) # ------------------------------- m4_defun([_LT_COMPILER_NO_RTTI], [m4_require([_LT_TAG_COMPILER])dnl _LT_TAGVAR(lt_prog_compiler_no_builtin_flag, $1)= if test yes = "$GCC"; then case $cc_basename in nvcc*) _LT_TAGVAR(lt_prog_compiler_no_builtin_flag, $1)=' -Xcompiler -fno-builtin' ;; *) _LT_TAGVAR(lt_prog_compiler_no_builtin_flag, $1)=' -fno-builtin' ;; esac _LT_COMPILER_OPTION([if $compiler supports -fno-rtti -fno-exceptions], lt_cv_prog_compiler_rtti_exceptions, [-fno-rtti -fno-exceptions], [], [_LT_TAGVAR(lt_prog_compiler_no_builtin_flag, $1)="$_LT_TAGVAR(lt_prog_compiler_no_builtin_flag, $1) -fno-rtti -fno-exceptions"]) fi _LT_TAGDECL([no_builtin_flag], [lt_prog_compiler_no_builtin_flag], [1], [Compiler flag to turn off builtin functions]) ])# _LT_COMPILER_NO_RTTI # _LT_CMD_GLOBAL_SYMBOLS # ---------------------- m4_defun([_LT_CMD_GLOBAL_SYMBOLS], [AC_REQUIRE([AC_CANONICAL_HOST])dnl AC_REQUIRE([AC_PROG_CC])dnl AC_REQUIRE([AC_PROG_AWK])dnl AC_REQUIRE([LT_PATH_NM])dnl AC_REQUIRE([LT_PATH_LD])dnl m4_require([_LT_DECL_SED])dnl m4_require([_LT_DECL_EGREP])dnl m4_require([_LT_TAG_COMPILER])dnl # Check for command to grab the raw symbol name followed by C symbol from nm. AC_MSG_CHECKING([command to parse $NM output from $compiler object]) AC_CACHE_VAL([lt_cv_sys_global_symbol_pipe], [ # These are sane defaults that work on at least a few old systems. # [They come from Ultrix. What could be older than Ultrix?!! ;)] # Character class describing NM global symbol codes. symcode='[[BCDEGRST]]' # Regexp to match symbols that can be accessed directly from C. sympat='\([[_A-Za-z]][[_A-Za-z0-9]]*\)' # Define system-specific variables. case $host_os in aix*) symcode='[[BCDT]]' ;; cygwin* | mingw* | pw32* | cegcc*) symcode='[[ABCDGISTW]]' ;; hpux*) if test ia64 = "$host_cpu"; then symcode='[[ABCDEGRST]]' fi ;; irix* | nonstopux*) symcode='[[BCDEGRST]]' ;; osf*) symcode='[[BCDEGQRST]]' ;; solaris*) symcode='[[BDRT]]' ;; sco3.2v5*) symcode='[[DT]]' ;; sysv4.2uw2*) symcode='[[DT]]' ;; sysv5* | sco5v6* | unixware* | OpenUNIX*) symcode='[[ABDT]]' ;; sysv4) symcode='[[DFNSTU]]' ;; esac # If we're using GNU nm, then use its standard symbol codes. case `$NM -V 2>&1` in *GNU* | *'with BFD'*) symcode='[[ABCDGIRSTW]]' ;; esac if test "$lt_cv_nm_interface" = "MS dumpbin"; then # Gets list of data symbols to import. lt_cv_sys_global_symbol_to_import="sed -n -e 's/^I .* \(.*\)$/\1/p'" # Adjust the below global symbol transforms to fixup imported variables. lt_cdecl_hook=" -e 's/^I .* \(.*\)$/extern __declspec(dllimport) char \1;/p'" lt_c_name_hook=" -e 's/^I .* \(.*\)$/ {\"\1\", (void *) 0},/p'" lt_c_name_lib_hook="\ -e 's/^I .* \(lib.*\)$/ {\"\1\", (void *) 0},/p'\ -e 's/^I .* \(.*\)$/ {\"lib\1\", (void *) 0},/p'" else # Disable hooks by default. lt_cv_sys_global_symbol_to_import= lt_cdecl_hook= lt_c_name_hook= lt_c_name_lib_hook= fi # Transform an extracted symbol line into a proper C declaration. # Some systems (esp. on ia64) link data and code symbols differently, # so use this general approach. lt_cv_sys_global_symbol_to_cdecl="sed -n"\ $lt_cdecl_hook\ " -e 's/^T .* \(.*\)$/extern int \1();/p'"\ " -e 's/^$symcode$symcode* .* \(.*\)$/extern char \1;/p'" # Transform an extracted symbol line into symbol name and symbol address lt_cv_sys_global_symbol_to_c_name_address="sed -n"\ $lt_c_name_hook\ " -e 's/^: \(.*\) .*$/ {\"\1\", (void *) 0},/p'"\ " -e 's/^$symcode$symcode* .* \(.*\)$/ {\"\1\", (void *) \&\1},/p'" # Transform an extracted symbol line into symbol name with lib prefix and # symbol address. lt_cv_sys_global_symbol_to_c_name_address_lib_prefix="sed -n"\ $lt_c_name_lib_hook\ " -e 's/^: \(.*\) .*$/ {\"\1\", (void *) 0},/p'"\ " -e 's/^$symcode$symcode* .* \(lib.*\)$/ {\"\1\", (void *) \&\1},/p'"\ " -e 's/^$symcode$symcode* .* \(.*\)$/ {\"lib\1\", (void *) \&\1},/p'" # Handle CRLF in mingw tool chain opt_cr= case $build_os in mingw*) opt_cr=`$ECHO 'x\{0,1\}' | tr x '\015'` # option cr in regexp ;; esac # Try without a prefix underscore, then with it. for ac_symprfx in "" "_"; do # Transform symcode, sympat, and symprfx into a raw symbol and a C symbol. symxfrm="\\1 $ac_symprfx\\2 \\2" # Write the raw and C identifiers. if test "$lt_cv_nm_interface" = "MS dumpbin"; then # Fake it for dumpbin and say T for any non-static function, # D for any global variable and I for any imported variable. # Also find C++ and __fastcall symbols from MSVC++, # which start with @ or ?. lt_cv_sys_global_symbol_pipe="$AWK ['"\ " {last_section=section; section=\$ 3};"\ " /^COFF SYMBOL TABLE/{for(i in hide) delete hide[i]};"\ " /Section length .*#relocs.*(pick any)/{hide[last_section]=1};"\ " /^ *Symbol name *: /{split(\$ 0,sn,\":\"); si=substr(sn[2],2)};"\ " /^ *Type *: code/{print \"T\",si,substr(si,length(prfx))};"\ " /^ *Type *: data/{print \"I\",si,substr(si,length(prfx))};"\ " \$ 0!~/External *\|/{next};"\ " / 0+ UNDEF /{next}; / UNDEF \([^|]\)*()/{next};"\ " {if(hide[section]) next};"\ " {f=\"D\"}; \$ 0~/\(\).*\|/{f=\"T\"};"\ " {split(\$ 0,a,/\||\r/); split(a[2],s)};"\ " s[1]~/^[@?]/{print f,s[1],s[1]; next};"\ " s[1]~prfx {split(s[1],t,\"@\"); print f,t[1],substr(t[1],length(prfx))}"\ " ' prfx=^$ac_symprfx]" else lt_cv_sys_global_symbol_pipe="sed -n -e 's/^.*[[ ]]\($symcode$symcode*\)[[ ]][[ ]]*$ac_symprfx$sympat$opt_cr$/$symxfrm/p'" fi lt_cv_sys_global_symbol_pipe="$lt_cv_sys_global_symbol_pipe | sed '/ __gnu_lto/d'" # Check to see that the pipe works correctly. pipe_works=no rm -f conftest* cat > conftest.$ac_ext <<_LT_EOF #ifdef __cplusplus extern "C" { #endif char nm_test_var; void nm_test_func(void); void nm_test_func(void){} #ifdef __cplusplus } #endif int main(){nm_test_var='a';nm_test_func();return(0);} _LT_EOF if AC_TRY_EVAL(ac_compile); then # Now try to grab the symbols. nlist=conftest.nm if AC_TRY_EVAL(NM conftest.$ac_objext \| "$lt_cv_sys_global_symbol_pipe" \> $nlist) && test -s "$nlist"; then # Try sorting and uniquifying the output. if sort "$nlist" | uniq > "$nlist"T; then mv -f "$nlist"T "$nlist" else rm -f "$nlist"T fi # Make sure that we snagged all the symbols we need. if $GREP ' nm_test_var$' "$nlist" >/dev/null; then if $GREP ' nm_test_func$' "$nlist" >/dev/null; then cat <<_LT_EOF > conftest.$ac_ext /* Keep this code in sync between libtool.m4, ltmain, lt_system.h, and tests. */ #if defined _WIN32 || defined __CYGWIN__ || defined _WIN32_WCE /* DATA imports from DLLs on WIN32 can't be const, because runtime relocations are performed -- see ld's documentation on pseudo-relocs. */ # define LT@&t@_DLSYM_CONST #elif defined __osf__ /* This system does not cope well with relocations in const data. */ # define LT@&t@_DLSYM_CONST #else # define LT@&t@_DLSYM_CONST const #endif #ifdef __cplusplus extern "C" { #endif _LT_EOF # Now generate the symbol file. eval "$lt_cv_sys_global_symbol_to_cdecl"' < "$nlist" | $GREP -v main >> conftest.$ac_ext' cat <<_LT_EOF >> conftest.$ac_ext /* The mapping between symbol names and symbols. */ LT@&t@_DLSYM_CONST struct { const char *name; void *address; } lt__PROGRAM__LTX_preloaded_symbols[[]] = { { "@PROGRAM@", (void *) 0 }, _LT_EOF $SED "s/^$symcode$symcode* .* \(.*\)$/ {\"\1\", (void *) \&\1},/" < "$nlist" | $GREP -v main >> conftest.$ac_ext cat <<\_LT_EOF >> conftest.$ac_ext {0, (void *) 0} }; /* This works around a problem in FreeBSD linker */ #ifdef FREEBSD_WORKAROUND static const void *lt_preloaded_setup() { return lt__PROGRAM__LTX_preloaded_symbols; } #endif #ifdef __cplusplus } #endif _LT_EOF # Now try linking the two files. mv conftest.$ac_objext conftstm.$ac_objext lt_globsym_save_LIBS=$LIBS lt_globsym_save_CFLAGS=$CFLAGS LIBS=conftstm.$ac_objext CFLAGS="$CFLAGS$_LT_TAGVAR(lt_prog_compiler_no_builtin_flag, $1)" if AC_TRY_EVAL(ac_link) && test -s conftest$ac_exeext; then pipe_works=yes fi LIBS=$lt_globsym_save_LIBS CFLAGS=$lt_globsym_save_CFLAGS else echo "cannot find nm_test_func in $nlist" >&AS_MESSAGE_LOG_FD fi else echo "cannot find nm_test_var in $nlist" >&AS_MESSAGE_LOG_FD fi else echo "cannot run $lt_cv_sys_global_symbol_pipe" >&AS_MESSAGE_LOG_FD fi else echo "$progname: failed program was:" >&AS_MESSAGE_LOG_FD cat conftest.$ac_ext >&5 fi rm -rf conftest* conftst* # Do not use the global_symbol_pipe unless it works. if test yes = "$pipe_works"; then break else lt_cv_sys_global_symbol_pipe= fi done ]) if test -z "$lt_cv_sys_global_symbol_pipe"; then lt_cv_sys_global_symbol_to_cdecl= fi if test -z "$lt_cv_sys_global_symbol_pipe$lt_cv_sys_global_symbol_to_cdecl"; then AC_MSG_RESULT(failed) else AC_MSG_RESULT(ok) fi # Response file support. if test "$lt_cv_nm_interface" = "MS dumpbin"; then nm_file_list_spec='@' elif $NM --help 2>/dev/null | grep '[[@]]FILE' >/dev/null; then nm_file_list_spec='@' fi _LT_DECL([global_symbol_pipe], [lt_cv_sys_global_symbol_pipe], [1], [Take the output of nm and produce a listing of raw symbols and C names]) _LT_DECL([global_symbol_to_cdecl], [lt_cv_sys_global_symbol_to_cdecl], [1], [Transform the output of nm in a proper C declaration]) _LT_DECL([global_symbol_to_import], [lt_cv_sys_global_symbol_to_import], [1], [Transform the output of nm into a list of symbols to manually relocate]) _LT_DECL([global_symbol_to_c_name_address], [lt_cv_sys_global_symbol_to_c_name_address], [1], [Transform the output of nm in a C name address pair]) _LT_DECL([global_symbol_to_c_name_address_lib_prefix], [lt_cv_sys_global_symbol_to_c_name_address_lib_prefix], [1], [Transform the output of nm in a C name address pair when lib prefix is needed]) _LT_DECL([nm_interface], [lt_cv_nm_interface], [1], [The name lister interface]) _LT_DECL([], [nm_file_list_spec], [1], [Specify filename containing input files for $NM]) ]) # _LT_CMD_GLOBAL_SYMBOLS # _LT_COMPILER_PIC([TAGNAME]) # --------------------------- m4_defun([_LT_COMPILER_PIC], [m4_require([_LT_TAG_COMPILER])dnl _LT_TAGVAR(lt_prog_compiler_wl, $1)= _LT_TAGVAR(lt_prog_compiler_pic, $1)= _LT_TAGVAR(lt_prog_compiler_static, $1)= m4_if([$1], [CXX], [ # C++ specific cases for pic, static, wl, etc. if test yes = "$GXX"; then _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' _LT_TAGVAR(lt_prog_compiler_static, $1)='-static' case $host_os in aix*) # All AIX code is PIC. if test ia64 = "$host_cpu"; then # AIX 5 now supports IA64 processor _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' fi _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' ;; amigaos*) case $host_cpu in powerpc) # see comment about AmigaOS4 .so support _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' ;; m68k) # FIXME: we need at least 68020 code to build shared libraries, but # adding the '-m68020' flag to GCC prevents building anything better, # like '-m68040'. _LT_TAGVAR(lt_prog_compiler_pic, $1)='-m68020 -resident32 -malways-restore-a4' ;; esac ;; beos* | irix5* | irix6* | nonstopux* | osf3* | osf4* | osf5*) # PIC is the default for these OSes. ;; mingw* | cygwin* | os2* | pw32* | cegcc*) # This hack is so that the source file can tell whether it is being # built for inclusion in a dll (and should export symbols for example). # Although the cygwin gcc ignores -fPIC, still need this for old-style # (--disable-auto-import) libraries m4_if([$1], [GCJ], [], [_LT_TAGVAR(lt_prog_compiler_pic, $1)='-DDLL_EXPORT']) case $host_os in os2*) _LT_TAGVAR(lt_prog_compiler_static, $1)='$wl-static' ;; esac ;; darwin* | rhapsody*) # PIC is the default on this platform # Common symbols not allowed in MH_DYLIB files _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fno-common' ;; *djgpp*) # DJGPP does not support shared libraries at all _LT_TAGVAR(lt_prog_compiler_pic, $1)= ;; haiku*) # PIC is the default for Haiku. # The "-static" flag exists, but is broken. _LT_TAGVAR(lt_prog_compiler_static, $1)= ;; interix[[3-9]]*) # Interix 3.x gcc -fpic/-fPIC options generate broken code. # Instead, we relocate shared libraries at runtime. ;; sysv4*MP*) if test -d /usr/nec; then _LT_TAGVAR(lt_prog_compiler_pic, $1)=-Kconform_pic fi ;; hpux*) # PIC is the default for 64-bit PA HP-UX, but not for 32-bit # PA HP-UX. On IA64 HP-UX, PIC is the default but the pic flag # sets the default TLS model and affects inlining. case $host_cpu in hppa*64*) ;; *) _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' ;; esac ;; *qnx* | *nto*) # QNX uses GNU C++, but need to define -shared option too, otherwise # it will coredump. _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC -shared' ;; *) _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' ;; esac else case $host_os in aix[[4-9]]*) # All AIX code is PIC. if test ia64 = "$host_cpu"; then # AIX 5 now supports IA64 processor _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' else _LT_TAGVAR(lt_prog_compiler_static, $1)='-bnso -bI:/lib/syscalls.exp' fi ;; chorus*) case $cc_basename in cxch68*) # Green Hills C++ Compiler # _LT_TAGVAR(lt_prog_compiler_static, $1)="--no_auto_instantiation -u __main -u __premain -u _abort -r $COOL_DIR/lib/libOrb.a $MVME_DIR/lib/CC/libC.a $MVME_DIR/lib/classix/libcx.s.a" ;; esac ;; mingw* | cygwin* | os2* | pw32* | cegcc*) # This hack is so that the source file can tell whether it is being # built for inclusion in a dll (and should export symbols for example). m4_if([$1], [GCJ], [], [_LT_TAGVAR(lt_prog_compiler_pic, $1)='-DDLL_EXPORT']) ;; dgux*) case $cc_basename in ec++*) _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' ;; ghcx*) # Green Hills C++ Compiler _LT_TAGVAR(lt_prog_compiler_pic, $1)='-pic' ;; *) ;; esac ;; freebsd* | dragonfly*) # FreeBSD uses GNU C++ ;; hpux9* | hpux10* | hpux11*) case $cc_basename in CC*) _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' _LT_TAGVAR(lt_prog_compiler_static, $1)='$wl-a ${wl}archive' if test ia64 != "$host_cpu"; then _LT_TAGVAR(lt_prog_compiler_pic, $1)='+Z' fi ;; aCC*) _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' _LT_TAGVAR(lt_prog_compiler_static, $1)='$wl-a ${wl}archive' case $host_cpu in hppa*64*|ia64*) # +Z the default ;; *) _LT_TAGVAR(lt_prog_compiler_pic, $1)='+Z' ;; esac ;; *) ;; esac ;; interix*) # This is c89, which is MS Visual C++ (no shared libs) # Anyone wants to do a port? ;; irix5* | irix6* | nonstopux*) case $cc_basename in CC*) _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' _LT_TAGVAR(lt_prog_compiler_static, $1)='-non_shared' # CC pic flag -KPIC is the default. ;; *) ;; esac ;; linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*) case $cc_basename in KCC*) # KAI C++ Compiler _LT_TAGVAR(lt_prog_compiler_wl, $1)='--backend -Wl,' _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' ;; ecpc* ) # old Intel C++ for x86_64, which still supported -KPIC. _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' _LT_TAGVAR(lt_prog_compiler_static, $1)='-static' ;; icpc* ) # Intel C++, used to be incompatible with GCC. # ICC 10 doesn't accept -KPIC any more. _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' _LT_TAGVAR(lt_prog_compiler_static, $1)='-static' ;; pgCC* | pgcpp*) # Portland Group C++ compiler _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fpic' _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' ;; cxx*) # Compaq C++ # Make sure the PIC flag is empty. It appears that all Alpha # Linux and Compaq Tru64 Unix objects are PIC. _LT_TAGVAR(lt_prog_compiler_pic, $1)= _LT_TAGVAR(lt_prog_compiler_static, $1)='-non_shared' ;; xlc* | xlC* | bgxl[[cC]]* | mpixl[[cC]]*) # IBM XL 8.0, 9.0 on PPC and BlueGene _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' _LT_TAGVAR(lt_prog_compiler_pic, $1)='-qpic' _LT_TAGVAR(lt_prog_compiler_static, $1)='-qstaticlink' ;; *) case `$CC -V 2>&1 | sed 5q` in *Sun\ C*) # Sun C++ 5.9 _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Qoption ld ' ;; esac ;; esac ;; lynxos*) ;; m88k*) ;; mvs*) case $cc_basename in cxx*) _LT_TAGVAR(lt_prog_compiler_pic, $1)='-W c,exportall' ;; *) ;; esac ;; netbsd* | netbsdelf*-gnu) ;; *qnx* | *nto*) # QNX uses GNU C++, but need to define -shared option too, otherwise # it will coredump. _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC -shared' ;; osf3* | osf4* | osf5*) case $cc_basename in KCC*) _LT_TAGVAR(lt_prog_compiler_wl, $1)='--backend -Wl,' ;; RCC*) # Rational C++ 2.4.1 _LT_TAGVAR(lt_prog_compiler_pic, $1)='-pic' ;; cxx*) # Digital/Compaq C++ _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' # Make sure the PIC flag is empty. It appears that all Alpha # Linux and Compaq Tru64 Unix objects are PIC. _LT_TAGVAR(lt_prog_compiler_pic, $1)= _LT_TAGVAR(lt_prog_compiler_static, $1)='-non_shared' ;; *) ;; esac ;; psos*) ;; solaris*) case $cc_basename in CC* | sunCC*) # Sun C++ 4.2, 5.x and Centerline C++ _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Qoption ld ' ;; gcx*) # Green Hills C++ Compiler _LT_TAGVAR(lt_prog_compiler_pic, $1)='-PIC' ;; *) ;; esac ;; sunos4*) case $cc_basename in CC*) # Sun C++ 4.x _LT_TAGVAR(lt_prog_compiler_pic, $1)='-pic' _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' ;; lcc*) # Lucid _LT_TAGVAR(lt_prog_compiler_pic, $1)='-pic' ;; *) ;; esac ;; sysv5* | unixware* | sco3.2v5* | sco5v6* | OpenUNIX*) case $cc_basename in CC*) _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' ;; esac ;; tandem*) case $cc_basename in NCC*) # NonStop-UX NCC 3.20 _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' ;; *) ;; esac ;; vxworks*) ;; *) _LT_TAGVAR(lt_prog_compiler_can_build_shared, $1)=no ;; esac fi ], [ if test yes = "$GCC"; then _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' _LT_TAGVAR(lt_prog_compiler_static, $1)='-static' case $host_os in aix*) # All AIX code is PIC. if test ia64 = "$host_cpu"; then # AIX 5 now supports IA64 processor _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' fi _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' ;; amigaos*) case $host_cpu in powerpc) # see comment about AmigaOS4 .so support _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' ;; m68k) # FIXME: we need at least 68020 code to build shared libraries, but # adding the '-m68020' flag to GCC prevents building anything better, # like '-m68040'. _LT_TAGVAR(lt_prog_compiler_pic, $1)='-m68020 -resident32 -malways-restore-a4' ;; esac ;; beos* | irix5* | irix6* | nonstopux* | osf3* | osf4* | osf5*) # PIC is the default for these OSes. ;; mingw* | cygwin* | pw32* | os2* | cegcc*) # This hack is so that the source file can tell whether it is being # built for inclusion in a dll (and should export symbols for example). # Although the cygwin gcc ignores -fPIC, still need this for old-style # (--disable-auto-import) libraries m4_if([$1], [GCJ], [], [_LT_TAGVAR(lt_prog_compiler_pic, $1)='-DDLL_EXPORT']) case $host_os in os2*) _LT_TAGVAR(lt_prog_compiler_static, $1)='$wl-static' ;; esac ;; darwin* | rhapsody*) # PIC is the default on this platform # Common symbols not allowed in MH_DYLIB files _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fno-common' ;; haiku*) # PIC is the default for Haiku. # The "-static" flag exists, but is broken. _LT_TAGVAR(lt_prog_compiler_static, $1)= ;; hpux*) # PIC is the default for 64-bit PA HP-UX, but not for 32-bit # PA HP-UX. On IA64 HP-UX, PIC is the default but the pic flag # sets the default TLS model and affects inlining. case $host_cpu in hppa*64*) # +Z the default ;; *) _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' ;; esac ;; interix[[3-9]]*) # Interix 3.x gcc -fpic/-fPIC options generate broken code. # Instead, we relocate shared libraries at runtime. ;; msdosdjgpp*) # Just because we use GCC doesn't mean we suddenly get shared libraries # on systems that don't support them. _LT_TAGVAR(lt_prog_compiler_can_build_shared, $1)=no enable_shared=no ;; *nto* | *qnx*) # QNX uses GNU C++, but need to define -shared option too, otherwise # it will coredump. _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC -shared' ;; sysv4*MP*) if test -d /usr/nec; then _LT_TAGVAR(lt_prog_compiler_pic, $1)=-Kconform_pic fi ;; *) _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' ;; esac case $cc_basename in nvcc*) # Cuda Compiler Driver 2.2 _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Xlinker ' if test -n "$_LT_TAGVAR(lt_prog_compiler_pic, $1)"; then _LT_TAGVAR(lt_prog_compiler_pic, $1)="-Xcompiler $_LT_TAGVAR(lt_prog_compiler_pic, $1)" fi ;; esac else # PORTME Check for flag to pass linker flags through the system compiler. case $host_os in aix*) _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' if test ia64 = "$host_cpu"; then # AIX 5 now supports IA64 processor _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' else _LT_TAGVAR(lt_prog_compiler_static, $1)='-bnso -bI:/lib/syscalls.exp' fi ;; darwin* | rhapsody*) # PIC is the default on this platform # Common symbols not allowed in MH_DYLIB files _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fno-common' case $cc_basename in nagfor*) # NAG Fortran compiler _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,-Wl,,' _LT_TAGVAR(lt_prog_compiler_pic, $1)='-PIC' _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' ;; esac ;; mingw* | cygwin* | pw32* | os2* | cegcc*) # This hack is so that the source file can tell whether it is being # built for inclusion in a dll (and should export symbols for example). m4_if([$1], [GCJ], [], [_LT_TAGVAR(lt_prog_compiler_pic, $1)='-DDLL_EXPORT']) case $host_os in os2*) _LT_TAGVAR(lt_prog_compiler_static, $1)='$wl-static' ;; esac ;; hpux9* | hpux10* | hpux11*) _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' # PIC is the default for IA64 HP-UX and 64-bit HP-UX, but # not for PA HP-UX. case $host_cpu in hppa*64*|ia64*) # +Z the default ;; *) _LT_TAGVAR(lt_prog_compiler_pic, $1)='+Z' ;; esac # Is there a better lt_prog_compiler_static that works with the bundled CC? _LT_TAGVAR(lt_prog_compiler_static, $1)='$wl-a ${wl}archive' ;; irix5* | irix6* | nonstopux*) _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' # PIC (with -KPIC) is the default. _LT_TAGVAR(lt_prog_compiler_static, $1)='-non_shared' ;; linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*) case $cc_basename in # old Intel for x86_64, which still supported -KPIC. ecc*) _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' _LT_TAGVAR(lt_prog_compiler_static, $1)='-static' ;; # icc used to be incompatible with GCC. # ICC 10 doesn't accept -KPIC any more. icc* | ifort*) _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' _LT_TAGVAR(lt_prog_compiler_static, $1)='-static' ;; # Lahey Fortran 8.1. lf95*) _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' _LT_TAGVAR(lt_prog_compiler_pic, $1)='--shared' _LT_TAGVAR(lt_prog_compiler_static, $1)='--static' ;; nagfor*) # NAG Fortran compiler _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,-Wl,,' _LT_TAGVAR(lt_prog_compiler_pic, $1)='-PIC' _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' ;; tcc*) # Fabrice Bellard et al's Tiny C Compiler _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' _LT_TAGVAR(lt_prog_compiler_static, $1)='-static' ;; pgcc* | pgf77* | pgf90* | pgf95* | pgfortran*) # Portland Group compilers (*not* the Pentium gcc compiler, # which looks to be a dead project) _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fpic' _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' ;; ccc*) _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' # All Alpha code is PIC. _LT_TAGVAR(lt_prog_compiler_static, $1)='-non_shared' ;; xl* | bgxl* | bgf* | mpixl*) # IBM XL C 8.0/Fortran 10.1, 11.1 on PPC and BlueGene _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' _LT_TAGVAR(lt_prog_compiler_pic, $1)='-qpic' _LT_TAGVAR(lt_prog_compiler_static, $1)='-qstaticlink' ;; *) case `$CC -V 2>&1 | sed 5q` in *Sun\ Ceres\ Fortran* | *Sun*Fortran*\ [[1-7]].* | *Sun*Fortran*\ 8.[[0-3]]*) # Sun Fortran 8.3 passes all unrecognized flags to the linker _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' _LT_TAGVAR(lt_prog_compiler_wl, $1)='' ;; *Sun\ F* | *Sun*Fortran*) _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Qoption ld ' ;; *Sun\ C*) # Sun C 5.9 _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' ;; *Intel*\ [[CF]]*Compiler*) _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' _LT_TAGVAR(lt_prog_compiler_static, $1)='-static' ;; *Portland\ Group*) _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fpic' _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' ;; esac ;; esac ;; newsos6) _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' ;; *nto* | *qnx*) # QNX uses GNU C++, but need to define -shared option too, otherwise # it will coredump. _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC -shared' ;; osf3* | osf4* | osf5*) _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' # All OSF/1 code is PIC. _LT_TAGVAR(lt_prog_compiler_static, $1)='-non_shared' ;; rdos*) _LT_TAGVAR(lt_prog_compiler_static, $1)='-non_shared' ;; solaris*) _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' case $cc_basename in f77* | f90* | f95* | sunf77* | sunf90* | sunf95*) _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Qoption ld ';; *) _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,';; esac ;; sunos4*) _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Qoption ld ' _LT_TAGVAR(lt_prog_compiler_pic, $1)='-PIC' _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' ;; sysv4 | sysv4.2uw2* | sysv4.3*) _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' ;; sysv4*MP*) if test -d /usr/nec; then _LT_TAGVAR(lt_prog_compiler_pic, $1)='-Kconform_pic' _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' fi ;; sysv5* | unixware* | sco3.2v5* | sco5v6* | OpenUNIX*) _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' ;; unicos*) _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' _LT_TAGVAR(lt_prog_compiler_can_build_shared, $1)=no ;; uts4*) _LT_TAGVAR(lt_prog_compiler_pic, $1)='-pic' _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' ;; *) _LT_TAGVAR(lt_prog_compiler_can_build_shared, $1)=no ;; esac fi ]) case $host_os in # For platforms that do not support PIC, -DPIC is meaningless: *djgpp*) _LT_TAGVAR(lt_prog_compiler_pic, $1)= ;; *) _LT_TAGVAR(lt_prog_compiler_pic, $1)="$_LT_TAGVAR(lt_prog_compiler_pic, $1)@&t@m4_if([$1],[],[ -DPIC],[m4_if([$1],[CXX],[ -DPIC],[])])" ;; esac AC_CACHE_CHECK([for $compiler option to produce PIC], [_LT_TAGVAR(lt_cv_prog_compiler_pic, $1)], [_LT_TAGVAR(lt_cv_prog_compiler_pic, $1)=$_LT_TAGVAR(lt_prog_compiler_pic, $1)]) _LT_TAGVAR(lt_prog_compiler_pic, $1)=$_LT_TAGVAR(lt_cv_prog_compiler_pic, $1) # # Check to make sure the PIC flag actually works. # if test -n "$_LT_TAGVAR(lt_prog_compiler_pic, $1)"; then _LT_COMPILER_OPTION([if $compiler PIC flag $_LT_TAGVAR(lt_prog_compiler_pic, $1) works], [_LT_TAGVAR(lt_cv_prog_compiler_pic_works, $1)], [$_LT_TAGVAR(lt_prog_compiler_pic, $1)@&t@m4_if([$1],[],[ -DPIC],[m4_if([$1],[CXX],[ -DPIC],[])])], [], [case $_LT_TAGVAR(lt_prog_compiler_pic, $1) in "" | " "*) ;; *) _LT_TAGVAR(lt_prog_compiler_pic, $1)=" $_LT_TAGVAR(lt_prog_compiler_pic, $1)" ;; esac], [_LT_TAGVAR(lt_prog_compiler_pic, $1)= _LT_TAGVAR(lt_prog_compiler_can_build_shared, $1)=no]) fi _LT_TAGDECL([pic_flag], [lt_prog_compiler_pic], [1], [Additional compiler flags for building library objects]) _LT_TAGDECL([wl], [lt_prog_compiler_wl], [1], [How to pass a linker flag through the compiler]) # # Check to make sure the static flag actually works. # wl=$_LT_TAGVAR(lt_prog_compiler_wl, $1) eval lt_tmp_static_flag=\"$_LT_TAGVAR(lt_prog_compiler_static, $1)\" _LT_LINKER_OPTION([if $compiler static flag $lt_tmp_static_flag works], _LT_TAGVAR(lt_cv_prog_compiler_static_works, $1), $lt_tmp_static_flag, [], [_LT_TAGVAR(lt_prog_compiler_static, $1)=]) _LT_TAGDECL([link_static_flag], [lt_prog_compiler_static], [1], [Compiler flag to prevent dynamic linking]) ])# _LT_COMPILER_PIC # _LT_LINKER_SHLIBS([TAGNAME]) # ---------------------------- # See if the linker supports building shared libraries. m4_defun([_LT_LINKER_SHLIBS], [AC_REQUIRE([LT_PATH_LD])dnl AC_REQUIRE([LT_PATH_NM])dnl m4_require([_LT_PATH_MANIFEST_TOOL])dnl m4_require([_LT_FILEUTILS_DEFAULTS])dnl m4_require([_LT_DECL_EGREP])dnl m4_require([_LT_DECL_SED])dnl m4_require([_LT_CMD_GLOBAL_SYMBOLS])dnl m4_require([_LT_TAG_COMPILER])dnl AC_MSG_CHECKING([whether the $compiler linker ($LD) supports shared libraries]) m4_if([$1], [CXX], [ _LT_TAGVAR(export_symbols_cmds, $1)='$NM $libobjs $convenience | $global_symbol_pipe | $SED '\''s/.* //'\'' | sort | uniq > $export_symbols' _LT_TAGVAR(exclude_expsyms, $1)=['_GLOBAL_OFFSET_TABLE_|_GLOBAL__F[ID]_.*'] case $host_os in aix[[4-9]]*) # If we're using GNU nm, then we don't want the "-C" option. # -C means demangle to GNU nm, but means don't demangle to AIX nm. # Without the "-l" option, or with the "-B" option, AIX nm treats # weak defined symbols like other global defined symbols, whereas # GNU nm marks them as "W". # While the 'weak' keyword is ignored in the Export File, we need # it in the Import File for the 'aix-soname' feature, so we have # to replace the "-B" option with "-P" for AIX nm. if $NM -V 2>&1 | $GREP 'GNU' > /dev/null; then _LT_TAGVAR(export_symbols_cmds, $1)='$NM -Bpg $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W")) && ([substr](\$ 3,1,1) != ".")) { if (\$ 2 == "W") { print \$ 3 " weak" } else { print \$ 3 } } }'\'' | sort -u > $export_symbols' else _LT_TAGVAR(export_symbols_cmds, $1)='`func_echo_all $NM | $SED -e '\''s/B\([[^B]]*\)$/P\1/'\''` -PCpgl $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) && ([substr](\$ 1,1,1) != ".")) { if ((\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) { print \$ 1 " weak" } else { print \$ 1 } } }'\'' | sort -u > $export_symbols' fi ;; pw32*) _LT_TAGVAR(export_symbols_cmds, $1)=$ltdll_cmds ;; cygwin* | mingw* | cegcc*) case $cc_basename in cl*) _LT_TAGVAR(exclude_expsyms, $1)='_NULL_IMPORT_DESCRIPTOR|_IMPORT_DESCRIPTOR_.*' ;; *) _LT_TAGVAR(export_symbols_cmds, $1)='$NM $libobjs $convenience | $global_symbol_pipe | $SED -e '\''/^[[BCDGRS]][[ ]]/s/.*[[ ]]\([[^ ]]*\)/\1 DATA/;s/^.*[[ ]]__nm__\([[^ ]]*\)[[ ]][[^ ]]*/\1 DATA/;/^I[[ ]]/d;/^[[AITW]][[ ]]/s/.* //'\'' | sort | uniq > $export_symbols' _LT_TAGVAR(exclude_expsyms, $1)=['[_]+GLOBAL_OFFSET_TABLE_|[_]+GLOBAL__[FID]_.*|[_]+head_[A-Za-z0-9_]+_dll|[A-Za-z0-9_]+_dll_iname'] ;; esac ;; linux* | k*bsd*-gnu | gnu*) _LT_TAGVAR(link_all_deplibs, $1)=no ;; *) _LT_TAGVAR(export_symbols_cmds, $1)='$NM $libobjs $convenience | $global_symbol_pipe | $SED '\''s/.* //'\'' | sort | uniq > $export_symbols' ;; esac ], [ runpath_var= _LT_TAGVAR(allow_undefined_flag, $1)= _LT_TAGVAR(always_export_symbols, $1)=no _LT_TAGVAR(archive_cmds, $1)= _LT_TAGVAR(archive_expsym_cmds, $1)= _LT_TAGVAR(compiler_needs_object, $1)=no _LT_TAGVAR(enable_shared_with_static_runtimes, $1)=no _LT_TAGVAR(export_dynamic_flag_spec, $1)= _LT_TAGVAR(export_symbols_cmds, $1)='$NM $libobjs $convenience | $global_symbol_pipe | $SED '\''s/.* //'\'' | sort | uniq > $export_symbols' _LT_TAGVAR(hardcode_automatic, $1)=no _LT_TAGVAR(hardcode_direct, $1)=no _LT_TAGVAR(hardcode_direct_absolute, $1)=no _LT_TAGVAR(hardcode_libdir_flag_spec, $1)= _LT_TAGVAR(hardcode_libdir_separator, $1)= _LT_TAGVAR(hardcode_minus_L, $1)=no _LT_TAGVAR(hardcode_shlibpath_var, $1)=unsupported _LT_TAGVAR(inherit_rpath, $1)=no _LT_TAGVAR(link_all_deplibs, $1)=unknown _LT_TAGVAR(module_cmds, $1)= _LT_TAGVAR(module_expsym_cmds, $1)= _LT_TAGVAR(old_archive_from_new_cmds, $1)= _LT_TAGVAR(old_archive_from_expsyms_cmds, $1)= _LT_TAGVAR(thread_safe_flag_spec, $1)= _LT_TAGVAR(whole_archive_flag_spec, $1)= # include_expsyms should be a list of space-separated symbols to be *always* # included in the symbol list _LT_TAGVAR(include_expsyms, $1)= # exclude_expsyms can be an extended regexp of symbols to exclude # it will be wrapped by ' (' and ')$', so one must not match beginning or # end of line. Example: 'a|bc|.*d.*' will exclude the symbols 'a' and 'bc', # as well as any symbol that contains 'd'. _LT_TAGVAR(exclude_expsyms, $1)=['_GLOBAL_OFFSET_TABLE_|_GLOBAL__F[ID]_.*'] # Although _GLOBAL_OFFSET_TABLE_ is a valid symbol C name, most a.out # platforms (ab)use it in PIC code, but their linkers get confused if # the symbol is explicitly referenced. Since portable code cannot # rely on this symbol name, it's probably fine to never include it in # preloaded symbol tables. # Exclude shared library initialization/finalization symbols. dnl Note also adjust exclude_expsyms for C++ above. extract_expsyms_cmds= case $host_os in cygwin* | mingw* | pw32* | cegcc*) # FIXME: the MSVC++ port hasn't been tested in a loooong time # When not using gcc, we currently assume that we are using # Microsoft Visual C++. if test yes != "$GCC"; then with_gnu_ld=no fi ;; interix*) # we just hope/assume this is gcc and not c89 (= MSVC++) with_gnu_ld=yes ;; openbsd* | bitrig*) with_gnu_ld=no ;; linux* | k*bsd*-gnu | gnu*) _LT_TAGVAR(link_all_deplibs, $1)=no ;; esac _LT_TAGVAR(ld_shlibs, $1)=yes # On some targets, GNU ld is compatible enough with the native linker # that we're better off using the native interface for both. lt_use_gnu_ld_interface=no if test yes = "$with_gnu_ld"; then case $host_os in aix*) # The AIX port of GNU ld has always aspired to compatibility # with the native linker. However, as the warning in the GNU ld # block says, versions before 2.19.5* couldn't really create working # shared libraries, regardless of the interface used. case `$LD -v 2>&1` in *\ \(GNU\ Binutils\)\ 2.19.5*) ;; *\ \(GNU\ Binutils\)\ 2.[[2-9]]*) ;; *\ \(GNU\ Binutils\)\ [[3-9]]*) ;; *) lt_use_gnu_ld_interface=yes ;; esac ;; *) lt_use_gnu_ld_interface=yes ;; esac fi if test yes = "$lt_use_gnu_ld_interface"; then # If archive_cmds runs LD, not CC, wlarc should be empty wlarc='$wl' # Set some defaults for GNU ld with shared library support. These # are reset later if shared libraries are not supported. Putting them # here allows them to be overridden if necessary. runpath_var=LD_RUN_PATH _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl--export-dynamic' # ancient GNU ld didn't support --whole-archive et. al. if $LD --help 2>&1 | $GREP 'no-whole-archive' > /dev/null; then _LT_TAGVAR(whole_archive_flag_spec, $1)=$wlarc'--whole-archive$convenience '$wlarc'--no-whole-archive' else _LT_TAGVAR(whole_archive_flag_spec, $1)= fi supports_anon_versioning=no case `$LD -v | $SED -e 's/([^)]\+)\s\+//' 2>&1` in *GNU\ gold*) supports_anon_versioning=yes ;; *\ [[01]].* | *\ 2.[[0-9]].* | *\ 2.10.*) ;; # catch versions < 2.11 *\ 2.11.93.0.2\ *) supports_anon_versioning=yes ;; # RH7.3 ... *\ 2.11.92.0.12\ *) supports_anon_versioning=yes ;; # Mandrake 8.2 ... *\ 2.11.*) ;; # other 2.11 versions *) supports_anon_versioning=yes ;; esac # See if GNU ld supports shared libraries. case $host_os in aix[[3-9]]*) # On AIX/PPC, the GNU linker is very broken if test ia64 != "$host_cpu"; then _LT_TAGVAR(ld_shlibs, $1)=no cat <<_LT_EOF 1>&2 *** Warning: the GNU linker, at least up to release 2.19, is reported *** to be unable to reliably create shared libraries on AIX. *** Therefore, libtool is disabling shared libraries support. If you *** really care for shared libraries, you may want to install binutils *** 2.20 or above, or modify your PATH so that a non-GNU linker is found. *** You will then need to restart the configuration process. _LT_EOF fi ;; amigaos*) case $host_cpu in powerpc) # see comment about AmigaOS4 .so support _LT_TAGVAR(archive_cmds, $1)='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' _LT_TAGVAR(archive_expsym_cmds, $1)='' ;; m68k) _LT_TAGVAR(archive_cmds, $1)='$RM $output_objdir/a2ixlibrary.data~$ECHO "#define NAME $libname" > $output_objdir/a2ixlibrary.data~$ECHO "#define LIBRARY_ID 1" >> $output_objdir/a2ixlibrary.data~$ECHO "#define VERSION $major" >> $output_objdir/a2ixlibrary.data~$ECHO "#define REVISION $revision" >> $output_objdir/a2ixlibrary.data~$AR $AR_FLAGS $lib $libobjs~$RANLIB $lib~(cd $output_objdir && a2ixlibrary -32)' _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' _LT_TAGVAR(hardcode_minus_L, $1)=yes ;; esac ;; beos*) if $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then _LT_TAGVAR(allow_undefined_flag, $1)=unsupported # Joseph Beckenbach says some releases of gcc # support --undefined. This deserves some investigation. FIXME _LT_TAGVAR(archive_cmds, $1)='$CC -nostart $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' else _LT_TAGVAR(ld_shlibs, $1)=no fi ;; cygwin* | mingw* | pw32* | cegcc*) # _LT_TAGVAR(hardcode_libdir_flag_spec, $1) is actually meaningless, # as there is no search path for DLLs. _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl--export-all-symbols' _LT_TAGVAR(allow_undefined_flag, $1)=unsupported _LT_TAGVAR(always_export_symbols, $1)=no _LT_TAGVAR(enable_shared_with_static_runtimes, $1)=yes _LT_TAGVAR(export_symbols_cmds, $1)='$NM $libobjs $convenience | $global_symbol_pipe | $SED -e '\''/^[[BCDGRS]][[ ]]/s/.*[[ ]]\([[^ ]]*\)/\1 DATA/;s/^.*[[ ]]__nm__\([[^ ]]*\)[[ ]][[^ ]]*/\1 DATA/;/^I[[ ]]/d;/^[[AITW]][[ ]]/s/.* //'\'' | sort | uniq > $export_symbols' _LT_TAGVAR(exclude_expsyms, $1)=['[_]+GLOBAL_OFFSET_TABLE_|[_]+GLOBAL__[FID]_.*|[_]+head_[A-Za-z0-9_]+_dll|[A-Za-z0-9_]+_dll_iname'] if $LD --help 2>&1 | $GREP 'auto-import' > /dev/null; then _LT_TAGVAR(archive_cmds, $1)='$CC -shared $libobjs $deplibs $compiler_flags -o $output_objdir/$soname $wl--enable-auto-image-base -Xlinker --out-implib -Xlinker $lib' # If the export-symbols file already is a .def file, use it as # is; otherwise, prepend EXPORTS... _LT_TAGVAR(archive_expsym_cmds, $1)='if _LT_DLL_DEF_P([$export_symbols]); then cp $export_symbols $output_objdir/$soname.def; else echo EXPORTS > $output_objdir/$soname.def; cat $export_symbols >> $output_objdir/$soname.def; fi~ $CC -shared $output_objdir/$soname.def $libobjs $deplibs $compiler_flags -o $output_objdir/$soname $wl--enable-auto-image-base -Xlinker --out-implib -Xlinker $lib' else _LT_TAGVAR(ld_shlibs, $1)=no fi ;; haiku*) _LT_TAGVAR(archive_cmds, $1)='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' _LT_TAGVAR(link_all_deplibs, $1)=yes ;; os2*) _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' _LT_TAGVAR(hardcode_minus_L, $1)=yes _LT_TAGVAR(allow_undefined_flag, $1)=unsupported shrext_cmds=.dll _LT_TAGVAR(archive_cmds, $1)='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ $ECHO EXPORTS >> $output_objdir/$libname.def~ emxexp $libobjs | $SED /"_DLL_InitTerm"/d >> $output_objdir/$libname.def~ $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ emximp -o $lib $output_objdir/$libname.def' _LT_TAGVAR(archive_expsym_cmds, $1)='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ $ECHO EXPORTS >> $output_objdir/$libname.def~ prefix_cmds="$SED"~ if test EXPORTS = "`$SED 1q $export_symbols`"; then prefix_cmds="$prefix_cmds -e 1d"; fi~ prefix_cmds="$prefix_cmds -e \"s/^\(.*\)$/_\1/g\""~ cat $export_symbols | $prefix_cmds >> $output_objdir/$libname.def~ $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ emximp -o $lib $output_objdir/$libname.def' _LT_TAGVAR(old_archive_From_new_cmds, $1)='emximp -o $output_objdir/${libname}_dll.a $output_objdir/$libname.def' _LT_TAGVAR(enable_shared_with_static_runtimes, $1)=yes ;; interix[[3-9]]*) _LT_TAGVAR(hardcode_direct, $1)=no _LT_TAGVAR(hardcode_shlibpath_var, $1)=no _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath,$libdir' _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-E' # Hack: On Interix 3.x, we cannot compile PIC because of a broken gcc. # Instead, shared libraries are loaded at an image base (0x10000000 by # default) and relocated if they conflict, which is a slow very memory # consuming and fragmenting process. To avoid this, we pick a random, # 256 KiB-aligned image base between 0x50000000 and 0x6FFC0000 at link # time. Moving up from 0x10000000 also allows more sbrk(2) space. _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-h,$soname $wl--image-base,`expr ${RANDOM-$$} % 4096 / 2 \* 262144 + 1342177280` -o $lib' _LT_TAGVAR(archive_expsym_cmds, $1)='sed "s|^|_|" $export_symbols >$output_objdir/$soname.expsym~$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-h,$soname $wl--retain-symbols-file,$output_objdir/$soname.expsym $wl--image-base,`expr ${RANDOM-$$} % 4096 / 2 \* 262144 + 1342177280` -o $lib' ;; gnu* | linux* | tpf* | k*bsd*-gnu | kopensolaris*-gnu) tmp_diet=no if test linux-dietlibc = "$host_os"; then case $cc_basename in diet\ *) tmp_diet=yes;; # linux-dietlibc with static linking (!diet-dyn) esac fi if $LD --help 2>&1 | $EGREP ': supported targets:.* elf' > /dev/null \ && test no = "$tmp_diet" then tmp_addflag=' $pic_flag' tmp_sharedflag='-shared' case $cc_basename,$host_cpu in pgcc*) # Portland Group C compiler _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl--whole-archive`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' tmp_addflag=' $pic_flag' ;; pgf77* | pgf90* | pgf95* | pgfortran*) # Portland Group f77 and f90 compilers _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl--whole-archive`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' tmp_addflag=' $pic_flag -Mnomain' ;; ecc*,ia64* | icc*,ia64*) # Intel C compiler on ia64 tmp_addflag=' -i_dynamic' ;; efc*,ia64* | ifort*,ia64*) # Intel Fortran compiler on ia64 tmp_addflag=' -i_dynamic -nofor_main' ;; ifc* | ifort*) # Intel Fortran compiler tmp_addflag=' -nofor_main' ;; lf95*) # Lahey Fortran 8.1 _LT_TAGVAR(whole_archive_flag_spec, $1)= tmp_sharedflag='--shared' ;; nagfor*) # NAGFOR 5.3 tmp_sharedflag='-Wl,-shared' ;; xl[[cC]]* | bgxl[[cC]]* | mpixl[[cC]]*) # IBM XL C 8.0 on PPC (deal with xlf below) tmp_sharedflag='-qmkshrobj' tmp_addflag= ;; nvcc*) # Cuda Compiler Driver 2.2 _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl--whole-archive`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' _LT_TAGVAR(compiler_needs_object, $1)=yes ;; esac case `$CC -V 2>&1 | sed 5q` in *Sun\ C*) # Sun C 5.9 _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl--whole-archive`new_convenience=; for conv in $convenience\"\"; do test -z \"$conv\" || new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' _LT_TAGVAR(compiler_needs_object, $1)=yes tmp_sharedflag='-G' ;; *Sun\ F*) # Sun Fortran 8.3 tmp_sharedflag='-G' ;; esac _LT_TAGVAR(archive_cmds, $1)='$CC '"$tmp_sharedflag""$tmp_addflag"' $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' if test yes = "$supports_anon_versioning"; then _LT_TAGVAR(archive_expsym_cmds, $1)='echo "{ global:" > $output_objdir/$libname.ver~ cat $export_symbols | sed -e "s/\(.*\)/\1;/" >> $output_objdir/$libname.ver~ echo "local: *; };" >> $output_objdir/$libname.ver~ $CC '"$tmp_sharedflag""$tmp_addflag"' $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-version-script $wl$output_objdir/$libname.ver -o $lib' fi case $cc_basename in tcc*) _LT_TAGVAR(export_dynamic_flag_spec, $1)='-rdynamic' ;; xlf* | bgf* | bgxlf* | mpixlf*) # IBM XL Fortran 10.1 on PPC cannot create shared libs itself _LT_TAGVAR(whole_archive_flag_spec, $1)='--whole-archive$convenience --no-whole-archive' _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' _LT_TAGVAR(archive_cmds, $1)='$LD -shared $libobjs $deplibs $linker_flags -soname $soname -o $lib' if test yes = "$supports_anon_versioning"; then _LT_TAGVAR(archive_expsym_cmds, $1)='echo "{ global:" > $output_objdir/$libname.ver~ cat $export_symbols | sed -e "s/\(.*\)/\1;/" >> $output_objdir/$libname.ver~ echo "local: *; };" >> $output_objdir/$libname.ver~ $LD -shared $libobjs $deplibs $linker_flags -soname $soname -version-script $output_objdir/$libname.ver -o $lib' fi ;; esac else _LT_TAGVAR(ld_shlibs, $1)=no fi ;; netbsd* | netbsdelf*-gnu) if echo __ELF__ | $CC -E - | $GREP __ELF__ >/dev/null; then _LT_TAGVAR(archive_cmds, $1)='$LD -Bshareable $libobjs $deplibs $linker_flags -o $lib' wlarc= else _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' fi ;; solaris*) if $LD -v 2>&1 | $GREP 'BFD 2\.8' > /dev/null; then _LT_TAGVAR(ld_shlibs, $1)=no cat <<_LT_EOF 1>&2 *** Warning: The releases 2.8.* of the GNU linker cannot reliably *** create shared libraries on Solaris systems. Therefore, libtool *** is disabling shared libraries support. We urge you to upgrade GNU *** binutils to release 2.9.1 or newer. Another option is to modify *** your PATH or compiler configuration so that the native linker is *** used, and then restart. _LT_EOF elif $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' else _LT_TAGVAR(ld_shlibs, $1)=no fi ;; sysv5* | sco3.2v5* | sco5v6* | unixware* | OpenUNIX*) case `$LD -v 2>&1` in *\ [[01]].* | *\ 2.[[0-9]].* | *\ 2.1[[0-5]].*) _LT_TAGVAR(ld_shlibs, $1)=no cat <<_LT_EOF 1>&2 *** Warning: Releases of the GNU linker prior to 2.16.91.0.3 cannot *** reliably create shared libraries on SCO systems. Therefore, libtool *** is disabling shared libraries support. We urge you to upgrade GNU *** binutils to release 2.16.91.0.3 or newer. Another option is to modify *** your PATH or compiler configuration so that the native linker is *** used, and then restart. _LT_EOF ;; *) # For security reasons, it is highly recommended that you always # use absolute paths for naming shared libraries, and exclude the # DT_RUNPATH tag from executables and libraries. But doing so # requires that you compile everything twice, which is a pain. if $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' _LT_TAGVAR(archive_cmds, $1)='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' else _LT_TAGVAR(ld_shlibs, $1)=no fi ;; esac ;; sunos4*) _LT_TAGVAR(archive_cmds, $1)='$LD -assert pure-text -Bshareable -o $lib $libobjs $deplibs $linker_flags' wlarc= _LT_TAGVAR(hardcode_direct, $1)=yes _LT_TAGVAR(hardcode_shlibpath_var, $1)=no ;; *) if $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' else _LT_TAGVAR(ld_shlibs, $1)=no fi ;; esac if test no = "$_LT_TAGVAR(ld_shlibs, $1)"; then runpath_var= _LT_TAGVAR(hardcode_libdir_flag_spec, $1)= _LT_TAGVAR(export_dynamic_flag_spec, $1)= _LT_TAGVAR(whole_archive_flag_spec, $1)= fi else # PORTME fill in a description of your system's linker (not GNU ld) case $host_os in aix3*) _LT_TAGVAR(allow_undefined_flag, $1)=unsupported _LT_TAGVAR(always_export_symbols, $1)=yes _LT_TAGVAR(archive_expsym_cmds, $1)='$LD -o $output_objdir/$soname $libobjs $deplibs $linker_flags -bE:$export_symbols -T512 -H512 -bM:SRE~$AR $AR_FLAGS $lib $output_objdir/$soname' # Note: this linker hardcodes the directories in LIBPATH if there # are no directories specified by -L. _LT_TAGVAR(hardcode_minus_L, $1)=yes if test yes = "$GCC" && test -z "$lt_prog_compiler_static"; then # Neither direct hardcoding nor static linking is supported with a # broken collect2. _LT_TAGVAR(hardcode_direct, $1)=unsupported fi ;; aix[[4-9]]*) if test ia64 = "$host_cpu"; then # On IA64, the linker does run time linking by default, so we don't # have to do anything special. aix_use_runtimelinking=no exp_sym_flag='-Bexport' no_entry_flag= else # If we're using GNU nm, then we don't want the "-C" option. # -C means demangle to GNU nm, but means don't demangle to AIX nm. # Without the "-l" option, or with the "-B" option, AIX nm treats # weak defined symbols like other global defined symbols, whereas # GNU nm marks them as "W". # While the 'weak' keyword is ignored in the Export File, we need # it in the Import File for the 'aix-soname' feature, so we have # to replace the "-B" option with "-P" for AIX nm. if $NM -V 2>&1 | $GREP 'GNU' > /dev/null; then _LT_TAGVAR(export_symbols_cmds, $1)='$NM -Bpg $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W")) && ([substr](\$ 3,1,1) != ".")) { if (\$ 2 == "W") { print \$ 3 " weak" } else { print \$ 3 } } }'\'' | sort -u > $export_symbols' else _LT_TAGVAR(export_symbols_cmds, $1)='`func_echo_all $NM | $SED -e '\''s/B\([[^B]]*\)$/P\1/'\''` -PCpgl $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) && ([substr](\$ 1,1,1) != ".")) { if ((\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) { print \$ 1 " weak" } else { print \$ 1 } } }'\'' | sort -u > $export_symbols' fi aix_use_runtimelinking=no # Test if we are trying to use run time linking or normal # AIX style linking. If -brtl is somewhere in LDFLAGS, we # have runtime linking enabled, and use it for executables. # For shared libraries, we enable/disable runtime linking # depending on the kind of the shared library created - # when "with_aix_soname,aix_use_runtimelinking" is: # "aix,no" lib.a(lib.so.V) shared, rtl:no, for executables # "aix,yes" lib.so shared, rtl:yes, for executables # lib.a static archive # "both,no" lib.so.V(shr.o) shared, rtl:yes # lib.a(lib.so.V) shared, rtl:no, for executables # "both,yes" lib.so.V(shr.o) shared, rtl:yes, for executables # lib.a(lib.so.V) shared, rtl:no # "svr4,*" lib.so.V(shr.o) shared, rtl:yes, for executables # lib.a static archive case $host_os in aix4.[[23]]|aix4.[[23]].*|aix[[5-9]]*) for ld_flag in $LDFLAGS; do if (test x-brtl = "x$ld_flag" || test x-Wl,-brtl = "x$ld_flag"); then aix_use_runtimelinking=yes break fi done if test svr4,no = "$with_aix_soname,$aix_use_runtimelinking"; then # With aix-soname=svr4, we create the lib.so.V shared archives only, # so we don't have lib.a shared libs to link our executables. # We have to force runtime linking in this case. aix_use_runtimelinking=yes LDFLAGS="$LDFLAGS -Wl,-brtl" fi ;; esac exp_sym_flag='-bexport' no_entry_flag='-bnoentry' fi # When large executables or shared objects are built, AIX ld can # have problems creating the table of contents. If linking a library # or program results in "error TOC overflow" add -mminimal-toc to # CXXFLAGS/CFLAGS for g++/gcc. In the cases where that is not # enough to fix the problem, add -Wl,-bbigtoc to LDFLAGS. _LT_TAGVAR(archive_cmds, $1)='' _LT_TAGVAR(hardcode_direct, $1)=yes _LT_TAGVAR(hardcode_direct_absolute, $1)=yes _LT_TAGVAR(hardcode_libdir_separator, $1)=':' _LT_TAGVAR(link_all_deplibs, $1)=yes _LT_TAGVAR(file_list_spec, $1)='$wl-f,' case $with_aix_soname,$aix_use_runtimelinking in aix,*) ;; # traditional, no import file svr4,* | *,yes) # use import file # The Import File defines what to hardcode. _LT_TAGVAR(hardcode_direct, $1)=no _LT_TAGVAR(hardcode_direct_absolute, $1)=no ;; esac if test yes = "$GCC"; then case $host_os in aix4.[[012]]|aix4.[[012]].*) # We only want to do this on AIX 4.2 and lower, the check # below for broken collect2 doesn't work under 4.3+ collect2name=`$CC -print-prog-name=collect2` if test -f "$collect2name" && strings "$collect2name" | $GREP resolve_lib_name >/dev/null then # We have reworked collect2 : else # We have old collect2 _LT_TAGVAR(hardcode_direct, $1)=unsupported # It fails to find uninstalled libraries when the uninstalled # path is not listed in the libpath. Setting hardcode_minus_L # to unsupported forces relinking _LT_TAGVAR(hardcode_minus_L, $1)=yes _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' _LT_TAGVAR(hardcode_libdir_separator, $1)= fi ;; esac shared_flag='-shared' if test yes = "$aix_use_runtimelinking"; then shared_flag="$shared_flag "'$wl-G' fi # Need to ensure runtime linking is disabled for the traditional # shared library, or the linker may eventually find shared libraries # /with/ Import File - we do not want to mix them. shared_flag_aix='-shared' shared_flag_svr4='-shared $wl-G' else # not using gcc if test ia64 = "$host_cpu"; then # VisualAge C++, Version 5.5 for AIX 5L for IA-64, Beta 3 Release # chokes on -Wl,-G. The following line is correct: shared_flag='-G' else if test yes = "$aix_use_runtimelinking"; then shared_flag='$wl-G' else shared_flag='$wl-bM:SRE' fi shared_flag_aix='$wl-bM:SRE' shared_flag_svr4='$wl-G' fi fi _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-bexpall' # It seems that -bexpall does not export symbols beginning with # underscore (_), so it is better to generate a list of symbols to export. _LT_TAGVAR(always_export_symbols, $1)=yes if test aix,yes = "$with_aix_soname,$aix_use_runtimelinking"; then # Warning - without using the other runtime loading flags (-brtl), # -berok will link without error, but may produce a broken library. _LT_TAGVAR(allow_undefined_flag, $1)='-berok' # Determine the default libpath from the value encoded in an # empty executable. _LT_SYS_MODULE_PATH_AIX([$1]) _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-blibpath:$libdir:'"$aix_libpath" _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -o $output_objdir/$soname $libobjs $deplibs $wl'$no_entry_flag' $compiler_flags `if test -n "$allow_undefined_flag"; then func_echo_all "$wl$allow_undefined_flag"; else :; fi` $wl'$exp_sym_flag:\$export_symbols' '$shared_flag else if test ia64 = "$host_cpu"; then _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-R $libdir:/usr/lib:/lib' _LT_TAGVAR(allow_undefined_flag, $1)="-z nodefs" _LT_TAGVAR(archive_expsym_cmds, $1)="\$CC $shared_flag"' -o $output_objdir/$soname $libobjs $deplibs '"\$wl$no_entry_flag"' $compiler_flags $wl$allow_undefined_flag '"\$wl$exp_sym_flag:\$export_symbols" else # Determine the default libpath from the value encoded in an # empty executable. _LT_SYS_MODULE_PATH_AIX([$1]) _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-blibpath:$libdir:'"$aix_libpath" # Warning - without using the other run time loading flags, # -berok will link without error, but may produce a broken library. _LT_TAGVAR(no_undefined_flag, $1)=' $wl-bernotok' _LT_TAGVAR(allow_undefined_flag, $1)=' $wl-berok' if test yes = "$with_gnu_ld"; then # We only use this code for GNU lds that support --whole-archive. _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl--whole-archive$convenience $wl--no-whole-archive' else # Exported symbols can be pulled into shared objects from archives _LT_TAGVAR(whole_archive_flag_spec, $1)='$convenience' fi _LT_TAGVAR(archive_cmds_need_lc, $1)=yes _LT_TAGVAR(archive_expsym_cmds, $1)='$RM -r $output_objdir/$realname.d~$MKDIR $output_objdir/$realname.d' # -brtl affects multiple linker settings, -berok does not and is overridden later compiler_flags_filtered='`func_echo_all "$compiler_flags " | $SED -e "s%-brtl\\([[, ]]\\)%-berok\\1%g"`' if test svr4 != "$with_aix_soname"; then # This is similar to how AIX traditionally builds its shared libraries. _LT_TAGVAR(archive_expsym_cmds, $1)="$_LT_TAGVAR(archive_expsym_cmds, $1)"'~$CC '$shared_flag_aix' -o $output_objdir/$realname.d/$soname $libobjs $deplibs $wl-bnoentry '$compiler_flags_filtered'$wl-bE:$export_symbols$allow_undefined_flag~$AR $AR_FLAGS $output_objdir/$libname$release.a $output_objdir/$realname.d/$soname' fi if test aix != "$with_aix_soname"; then _LT_TAGVAR(archive_expsym_cmds, $1)="$_LT_TAGVAR(archive_expsym_cmds, $1)"'~$CC '$shared_flag_svr4' -o $output_objdir/$realname.d/$shared_archive_member_spec.o $libobjs $deplibs $wl-bnoentry '$compiler_flags_filtered'$wl-bE:$export_symbols$allow_undefined_flag~$STRIP -e $output_objdir/$realname.d/$shared_archive_member_spec.o~( func_echo_all "#! $soname($shared_archive_member_spec.o)"; if test shr_64 = "$shared_archive_member_spec"; then func_echo_all "# 64"; else func_echo_all "# 32"; fi; cat $export_symbols ) > $output_objdir/$realname.d/$shared_archive_member_spec.imp~$AR $AR_FLAGS $output_objdir/$soname $output_objdir/$realname.d/$shared_archive_member_spec.o $output_objdir/$realname.d/$shared_archive_member_spec.imp' else # used by -dlpreopen to get the symbols _LT_TAGVAR(archive_expsym_cmds, $1)="$_LT_TAGVAR(archive_expsym_cmds, $1)"'~$MV $output_objdir/$realname.d/$soname $output_objdir' fi _LT_TAGVAR(archive_expsym_cmds, $1)="$_LT_TAGVAR(archive_expsym_cmds, $1)"'~$RM -r $output_objdir/$realname.d' fi fi ;; amigaos*) case $host_cpu in powerpc) # see comment about AmigaOS4 .so support _LT_TAGVAR(archive_cmds, $1)='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' _LT_TAGVAR(archive_expsym_cmds, $1)='' ;; m68k) _LT_TAGVAR(archive_cmds, $1)='$RM $output_objdir/a2ixlibrary.data~$ECHO "#define NAME $libname" > $output_objdir/a2ixlibrary.data~$ECHO "#define LIBRARY_ID 1" >> $output_objdir/a2ixlibrary.data~$ECHO "#define VERSION $major" >> $output_objdir/a2ixlibrary.data~$ECHO "#define REVISION $revision" >> $output_objdir/a2ixlibrary.data~$AR $AR_FLAGS $lib $libobjs~$RANLIB $lib~(cd $output_objdir && a2ixlibrary -32)' _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' _LT_TAGVAR(hardcode_minus_L, $1)=yes ;; esac ;; bsdi[[45]]*) _LT_TAGVAR(export_dynamic_flag_spec, $1)=-rdynamic ;; cygwin* | mingw* | pw32* | cegcc*) # When not using gcc, we currently assume that we are using # Microsoft Visual C++. # hardcode_libdir_flag_spec is actually meaningless, as there is # no search path for DLLs. case $cc_basename in cl*) # Native MSVC _LT_TAGVAR(hardcode_libdir_flag_spec, $1)=' ' _LT_TAGVAR(allow_undefined_flag, $1)=unsupported _LT_TAGVAR(always_export_symbols, $1)=yes _LT_TAGVAR(file_list_spec, $1)='@' # Tell ltmain to make .lib files, not .a files. libext=lib # Tell ltmain to make .dll files, not .so files. shrext_cmds=.dll # FIXME: Setting linknames here is a bad hack. _LT_TAGVAR(archive_cmds, $1)='$CC -o $output_objdir/$soname $libobjs $compiler_flags $deplibs -Wl,-DLL,-IMPLIB:"$tool_output_objdir$libname.dll.lib"~linknames=' _LT_TAGVAR(archive_expsym_cmds, $1)='if _LT_DLL_DEF_P([$export_symbols]); then cp "$export_symbols" "$output_objdir/$soname.def"; echo "$tool_output_objdir$soname.def" > "$output_objdir/$soname.exp"; else $SED -e '\''s/^/-link -EXPORT:/'\'' < $export_symbols > $output_objdir/$soname.exp; fi~ $CC -o $tool_output_objdir$soname $libobjs $compiler_flags $deplibs "@$tool_output_objdir$soname.exp" -Wl,-DLL,-IMPLIB:"$tool_output_objdir$libname.dll.lib"~ linknames=' # The linker will not automatically build a static lib if we build a DLL. # _LT_TAGVAR(old_archive_from_new_cmds, $1)='true' _LT_TAGVAR(enable_shared_with_static_runtimes, $1)=yes _LT_TAGVAR(exclude_expsyms, $1)='_NULL_IMPORT_DESCRIPTOR|_IMPORT_DESCRIPTOR_.*' _LT_TAGVAR(export_symbols_cmds, $1)='$NM $libobjs $convenience | $global_symbol_pipe | $SED -e '\''/^[[BCDGRS]][[ ]]/s/.*[[ ]]\([[^ ]]*\)/\1,DATA/'\'' | $SED -e '\''/^[[AITW]][[ ]]/s/.*[[ ]]//'\'' | sort | uniq > $export_symbols' # Don't use ranlib _LT_TAGVAR(old_postinstall_cmds, $1)='chmod 644 $oldlib' _LT_TAGVAR(postlink_cmds, $1)='lt_outputfile="@OUTPUT@"~ lt_tool_outputfile="@TOOL_OUTPUT@"~ case $lt_outputfile in *.exe|*.EXE) ;; *) lt_outputfile=$lt_outputfile.exe lt_tool_outputfile=$lt_tool_outputfile.exe ;; esac~ if test : != "$MANIFEST_TOOL" && test -f "$lt_outputfile.manifest"; then $MANIFEST_TOOL -manifest "$lt_tool_outputfile.manifest" -outputresource:"$lt_tool_outputfile" || exit 1; $RM "$lt_outputfile.manifest"; fi' ;; *) # Assume MSVC wrapper _LT_TAGVAR(hardcode_libdir_flag_spec, $1)=' ' _LT_TAGVAR(allow_undefined_flag, $1)=unsupported # Tell ltmain to make .lib files, not .a files. libext=lib # Tell ltmain to make .dll files, not .so files. shrext_cmds=.dll # FIXME: Setting linknames here is a bad hack. _LT_TAGVAR(archive_cmds, $1)='$CC -o $lib $libobjs $compiler_flags `func_echo_all "$deplibs" | $SED '\''s/ -lc$//'\''` -link -dll~linknames=' # The linker will automatically build a .lib file if we build a DLL. _LT_TAGVAR(old_archive_from_new_cmds, $1)='true' # FIXME: Should let the user specify the lib program. _LT_TAGVAR(old_archive_cmds, $1)='lib -OUT:$oldlib$oldobjs$old_deplibs' _LT_TAGVAR(enable_shared_with_static_runtimes, $1)=yes ;; esac ;; darwin* | rhapsody*) _LT_DARWIN_LINKER_FEATURES($1) ;; dgux*) _LT_TAGVAR(archive_cmds, $1)='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' _LT_TAGVAR(hardcode_shlibpath_var, $1)=no ;; # FreeBSD 2.2.[012] allows us to include c++rt0.o to get C++ constructor # support. Future versions do this automatically, but an explicit c++rt0.o # does not break anything, and helps significantly (at the cost of a little # extra space). freebsd2.2*) _LT_TAGVAR(archive_cmds, $1)='$LD -Bshareable -o $lib $libobjs $deplibs $linker_flags /usr/lib/c++rt0.o' _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-R$libdir' _LT_TAGVAR(hardcode_direct, $1)=yes _LT_TAGVAR(hardcode_shlibpath_var, $1)=no ;; # Unfortunately, older versions of FreeBSD 2 do not have this feature. freebsd2.*) _LT_TAGVAR(archive_cmds, $1)='$LD -Bshareable -o $lib $libobjs $deplibs $linker_flags' _LT_TAGVAR(hardcode_direct, $1)=yes _LT_TAGVAR(hardcode_minus_L, $1)=yes _LT_TAGVAR(hardcode_shlibpath_var, $1)=no ;; # FreeBSD 3 and greater uses gcc -shared to do shared libraries. freebsd* | dragonfly*) _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags' _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-R$libdir' _LT_TAGVAR(hardcode_direct, $1)=yes _LT_TAGVAR(hardcode_shlibpath_var, $1)=no ;; hpux9*) if test yes = "$GCC"; then _LT_TAGVAR(archive_cmds, $1)='$RM $output_objdir/$soname~$CC -shared $pic_flag $wl+b $wl$install_libdir -o $output_objdir/$soname $libobjs $deplibs $compiler_flags~test "x$output_objdir/$soname" = "x$lib" || mv $output_objdir/$soname $lib' else _LT_TAGVAR(archive_cmds, $1)='$RM $output_objdir/$soname~$LD -b +b $install_libdir -o $output_objdir/$soname $libobjs $deplibs $linker_flags~test "x$output_objdir/$soname" = "x$lib" || mv $output_objdir/$soname $lib' fi _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl+b $wl$libdir' _LT_TAGVAR(hardcode_libdir_separator, $1)=: _LT_TAGVAR(hardcode_direct, $1)=yes # hardcode_minus_L: Not really in the search PATH, # but as the default location of the library. _LT_TAGVAR(hardcode_minus_L, $1)=yes _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-E' ;; hpux10*) if test yes,no = "$GCC,$with_gnu_ld"; then _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $wl+h $wl$soname $wl+b $wl$install_libdir -o $lib $libobjs $deplibs $compiler_flags' else _LT_TAGVAR(archive_cmds, $1)='$LD -b +h $soname +b $install_libdir -o $lib $libobjs $deplibs $linker_flags' fi if test no = "$with_gnu_ld"; then _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl+b $wl$libdir' _LT_TAGVAR(hardcode_libdir_separator, $1)=: _LT_TAGVAR(hardcode_direct, $1)=yes _LT_TAGVAR(hardcode_direct_absolute, $1)=yes _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-E' # hardcode_minus_L: Not really in the search PATH, # but as the default location of the library. _LT_TAGVAR(hardcode_minus_L, $1)=yes fi ;; hpux11*) if test yes,no = "$GCC,$with_gnu_ld"; then case $host_cpu in hppa*64*) _LT_TAGVAR(archive_cmds, $1)='$CC -shared $wl+h $wl$soname -o $lib $libobjs $deplibs $compiler_flags' ;; ia64*) _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $wl+h $wl$soname $wl+nodefaultrpath -o $lib $libobjs $deplibs $compiler_flags' ;; *) _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $wl+h $wl$soname $wl+b $wl$install_libdir -o $lib $libobjs $deplibs $compiler_flags' ;; esac else case $host_cpu in hppa*64*) _LT_TAGVAR(archive_cmds, $1)='$CC -b $wl+h $wl$soname -o $lib $libobjs $deplibs $compiler_flags' ;; ia64*) _LT_TAGVAR(archive_cmds, $1)='$CC -b $wl+h $wl$soname $wl+nodefaultrpath -o $lib $libobjs $deplibs $compiler_flags' ;; *) m4_if($1, [], [ # Older versions of the 11.00 compiler do not understand -b yet # (HP92453-01 A.11.01.20 doesn't, HP92453-01 B.11.X.35175-35176.GP does) _LT_LINKER_OPTION([if $CC understands -b], _LT_TAGVAR(lt_cv_prog_compiler__b, $1), [-b], [_LT_TAGVAR(archive_cmds, $1)='$CC -b $wl+h $wl$soname $wl+b $wl$install_libdir -o $lib $libobjs $deplibs $compiler_flags'], [_LT_TAGVAR(archive_cmds, $1)='$LD -b +h $soname +b $install_libdir -o $lib $libobjs $deplibs $linker_flags'])], [_LT_TAGVAR(archive_cmds, $1)='$CC -b $wl+h $wl$soname $wl+b $wl$install_libdir -o $lib $libobjs $deplibs $compiler_flags']) ;; esac fi if test no = "$with_gnu_ld"; then _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl+b $wl$libdir' _LT_TAGVAR(hardcode_libdir_separator, $1)=: case $host_cpu in hppa*64*|ia64*) _LT_TAGVAR(hardcode_direct, $1)=no _LT_TAGVAR(hardcode_shlibpath_var, $1)=no ;; *) _LT_TAGVAR(hardcode_direct, $1)=yes _LT_TAGVAR(hardcode_direct_absolute, $1)=yes _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-E' # hardcode_minus_L: Not really in the search PATH, # but as the default location of the library. _LT_TAGVAR(hardcode_minus_L, $1)=yes ;; esac fi ;; irix5* | irix6* | nonstopux*) if test yes = "$GCC"; then _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations -o $lib' # Try to use the -exported_symbol ld option, if it does not # work, assume that -exports_file does not work either and # implicitly export all symbols. # This should be the same for all languages, so no per-tag cache variable. AC_CACHE_CHECK([whether the $host_os linker accepts -exported_symbol], [lt_cv_irix_exported_symbol], [save_LDFLAGS=$LDFLAGS LDFLAGS="$LDFLAGS -shared $wl-exported_symbol ${wl}foo $wl-update_registry $wl/dev/null" AC_LINK_IFELSE( [AC_LANG_SOURCE( [AC_LANG_CASE([C], [[int foo (void) { return 0; }]], [C++], [[int foo (void) { return 0; }]], [Fortran 77], [[ subroutine foo end]], [Fortran], [[ subroutine foo end]])])], [lt_cv_irix_exported_symbol=yes], [lt_cv_irix_exported_symbol=no]) LDFLAGS=$save_LDFLAGS]) if test yes = "$lt_cv_irix_exported_symbol"; then _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations $wl-exports_file $wl$export_symbols -o $lib' fi _LT_TAGVAR(link_all_deplibs, $1)=no else _LT_TAGVAR(archive_cmds, $1)='$CC -shared $libobjs $deplibs $compiler_flags -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib' _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $libobjs $deplibs $compiler_flags -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -exports_file $export_symbols -o $lib' fi _LT_TAGVAR(archive_cmds_need_lc, $1)='no' _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' _LT_TAGVAR(hardcode_libdir_separator, $1)=: _LT_TAGVAR(inherit_rpath, $1)=yes _LT_TAGVAR(link_all_deplibs, $1)=yes ;; linux*) case $cc_basename in tcc*) # Fabrice Bellard et al's Tiny C Compiler _LT_TAGVAR(ld_shlibs, $1)=yes _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags' ;; esac ;; netbsd* | netbsdelf*-gnu) if echo __ELF__ | $CC -E - | $GREP __ELF__ >/dev/null; then _LT_TAGVAR(archive_cmds, $1)='$LD -Bshareable -o $lib $libobjs $deplibs $linker_flags' # a.out else _LT_TAGVAR(archive_cmds, $1)='$LD -shared -o $lib $libobjs $deplibs $linker_flags' # ELF fi _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-R$libdir' _LT_TAGVAR(hardcode_direct, $1)=yes _LT_TAGVAR(hardcode_shlibpath_var, $1)=no ;; newsos6) _LT_TAGVAR(archive_cmds, $1)='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' _LT_TAGVAR(hardcode_direct, $1)=yes _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' _LT_TAGVAR(hardcode_libdir_separator, $1)=: _LT_TAGVAR(hardcode_shlibpath_var, $1)=no ;; *nto* | *qnx*) ;; openbsd* | bitrig*) if test -f /usr/libexec/ld.so; then _LT_TAGVAR(hardcode_direct, $1)=yes _LT_TAGVAR(hardcode_shlibpath_var, $1)=no _LT_TAGVAR(hardcode_direct_absolute, $1)=yes if test -z "`echo __ELF__ | $CC -E - | $GREP __ELF__`"; then _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags' _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags $wl-retain-symbols-file,$export_symbols' _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath,$libdir' _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-E' else _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags' _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath,$libdir' fi else _LT_TAGVAR(ld_shlibs, $1)=no fi ;; os2*) _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' _LT_TAGVAR(hardcode_minus_L, $1)=yes _LT_TAGVAR(allow_undefined_flag, $1)=unsupported shrext_cmds=.dll _LT_TAGVAR(archive_cmds, $1)='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ $ECHO EXPORTS >> $output_objdir/$libname.def~ emxexp $libobjs | $SED /"_DLL_InitTerm"/d >> $output_objdir/$libname.def~ $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ emximp -o $lib $output_objdir/$libname.def' _LT_TAGVAR(archive_expsym_cmds, $1)='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ $ECHO EXPORTS >> $output_objdir/$libname.def~ prefix_cmds="$SED"~ if test EXPORTS = "`$SED 1q $export_symbols`"; then prefix_cmds="$prefix_cmds -e 1d"; fi~ prefix_cmds="$prefix_cmds -e \"s/^\(.*\)$/_\1/g\""~ cat $export_symbols | $prefix_cmds >> $output_objdir/$libname.def~ $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ emximp -o $lib $output_objdir/$libname.def' _LT_TAGVAR(old_archive_From_new_cmds, $1)='emximp -o $output_objdir/${libname}_dll.a $output_objdir/$libname.def' _LT_TAGVAR(enable_shared_with_static_runtimes, $1)=yes ;; osf3*) if test yes = "$GCC"; then _LT_TAGVAR(allow_undefined_flag, $1)=' $wl-expect_unresolved $wl\*' _LT_TAGVAR(archive_cmds, $1)='$CC -shared$allow_undefined_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations -o $lib' else _LT_TAGVAR(allow_undefined_flag, $1)=' -expect_unresolved \*' _LT_TAGVAR(archive_cmds, $1)='$CC -shared$allow_undefined_flag $libobjs $deplibs $compiler_flags -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib' fi _LT_TAGVAR(archive_cmds_need_lc, $1)='no' _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' _LT_TAGVAR(hardcode_libdir_separator, $1)=: ;; osf4* | osf5*) # as osf3* with the addition of -msym flag if test yes = "$GCC"; then _LT_TAGVAR(allow_undefined_flag, $1)=' $wl-expect_unresolved $wl\*' _LT_TAGVAR(archive_cmds, $1)='$CC -shared$allow_undefined_flag $pic_flag $libobjs $deplibs $compiler_flags $wl-msym $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations -o $lib' _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' else _LT_TAGVAR(allow_undefined_flag, $1)=' -expect_unresolved \*' _LT_TAGVAR(archive_cmds, $1)='$CC -shared$allow_undefined_flag $libobjs $deplibs $compiler_flags -msym -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib' _LT_TAGVAR(archive_expsym_cmds, $1)='for i in `cat $export_symbols`; do printf "%s %s\\n" -exported_symbol "\$i" >> $lib.exp; done; printf "%s\\n" "-hidden">> $lib.exp~ $CC -shared$allow_undefined_flag $wl-input $wl$lib.exp $compiler_flags $libobjs $deplibs -soname $soname `test -n "$verstring" && $ECHO "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib~$RM $lib.exp' # Both c and cxx compiler support -rpath directly _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-rpath $libdir' fi _LT_TAGVAR(archive_cmds_need_lc, $1)='no' _LT_TAGVAR(hardcode_libdir_separator, $1)=: ;; solaris*) _LT_TAGVAR(no_undefined_flag, $1)=' -z defs' if test yes = "$GCC"; then wlarc='$wl' _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $wl-z ${wl}text $wl-h $wl$soname -o $lib $libobjs $deplibs $compiler_flags' _LT_TAGVAR(archive_expsym_cmds, $1)='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ $CC -shared $pic_flag $wl-z ${wl}text $wl-M $wl$lib.exp $wl-h $wl$soname -o $lib $libobjs $deplibs $compiler_flags~$RM $lib.exp' else case `$CC -V 2>&1` in *"Compilers 5.0"*) wlarc='' _LT_TAGVAR(archive_cmds, $1)='$LD -G$allow_undefined_flag -h $soname -o $lib $libobjs $deplibs $linker_flags' _LT_TAGVAR(archive_expsym_cmds, $1)='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ $LD -G$allow_undefined_flag -M $lib.exp -h $soname -o $lib $libobjs $deplibs $linker_flags~$RM $lib.exp' ;; *) wlarc='$wl' _LT_TAGVAR(archive_cmds, $1)='$CC -G$allow_undefined_flag -h $soname -o $lib $libobjs $deplibs $compiler_flags' _LT_TAGVAR(archive_expsym_cmds, $1)='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ $CC -G$allow_undefined_flag -M $lib.exp -h $soname -o $lib $libobjs $deplibs $compiler_flags~$RM $lib.exp' ;; esac fi _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-R$libdir' _LT_TAGVAR(hardcode_shlibpath_var, $1)=no case $host_os in solaris2.[[0-5]] | solaris2.[[0-5]].*) ;; *) # The compiler driver will combine and reorder linker options, # but understands '-z linker_flag'. GCC discards it without '$wl', # but is careful enough not to reorder. # Supported since Solaris 2.6 (maybe 2.5.1?) if test yes = "$GCC"; then _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl-z ${wl}allextract$convenience $wl-z ${wl}defaultextract' else _LT_TAGVAR(whole_archive_flag_spec, $1)='-z allextract$convenience -z defaultextract' fi ;; esac _LT_TAGVAR(link_all_deplibs, $1)=yes ;; sunos4*) if test sequent = "$host_vendor"; then # Use $CC to link under sequent, because it throws in some extra .o # files that make .init and .fini sections work. _LT_TAGVAR(archive_cmds, $1)='$CC -G $wl-h $soname -o $lib $libobjs $deplibs $compiler_flags' else _LT_TAGVAR(archive_cmds, $1)='$LD -assert pure-text -Bstatic -o $lib $libobjs $deplibs $linker_flags' fi _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' _LT_TAGVAR(hardcode_direct, $1)=yes _LT_TAGVAR(hardcode_minus_L, $1)=yes _LT_TAGVAR(hardcode_shlibpath_var, $1)=no ;; sysv4) case $host_vendor in sni) _LT_TAGVAR(archive_cmds, $1)='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' _LT_TAGVAR(hardcode_direct, $1)=yes # is this really true??? ;; siemens) ## LD is ld it makes a PLAMLIB ## CC just makes a GrossModule. _LT_TAGVAR(archive_cmds, $1)='$LD -G -o $lib $libobjs $deplibs $linker_flags' _LT_TAGVAR(reload_cmds, $1)='$CC -r -o $output$reload_objs' _LT_TAGVAR(hardcode_direct, $1)=no ;; motorola) _LT_TAGVAR(archive_cmds, $1)='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' _LT_TAGVAR(hardcode_direct, $1)=no #Motorola manual says yes, but my tests say they lie ;; esac runpath_var='LD_RUN_PATH' _LT_TAGVAR(hardcode_shlibpath_var, $1)=no ;; sysv4.3*) _LT_TAGVAR(archive_cmds, $1)='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' _LT_TAGVAR(hardcode_shlibpath_var, $1)=no _LT_TAGVAR(export_dynamic_flag_spec, $1)='-Bexport' ;; sysv4*MP*) if test -d /usr/nec; then _LT_TAGVAR(archive_cmds, $1)='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' _LT_TAGVAR(hardcode_shlibpath_var, $1)=no runpath_var=LD_RUN_PATH hardcode_runpath_var=yes _LT_TAGVAR(ld_shlibs, $1)=yes fi ;; sysv4*uw2* | sysv5OpenUNIX* | sysv5UnixWare7.[[01]].[[10]]* | unixware7* | sco3.2v5.0.[[024]]*) _LT_TAGVAR(no_undefined_flag, $1)='$wl-z,text' _LT_TAGVAR(archive_cmds_need_lc, $1)=no _LT_TAGVAR(hardcode_shlibpath_var, $1)=no runpath_var='LD_RUN_PATH' if test yes = "$GCC"; then _LT_TAGVAR(archive_cmds, $1)='$CC -shared $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' else _LT_TAGVAR(archive_cmds, $1)='$CC -G $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -G $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' fi ;; sysv5* | sco3.2v5* | sco5v6*) # Note: We CANNOT use -z defs as we might desire, because we do not # link with -lc, and that would cause any symbols used from libc to # always be unresolved, which means just about no library would # ever link correctly. If we're not using GNU ld we use -z text # though, which does catch some bad symbols but isn't as heavy-handed # as -z defs. _LT_TAGVAR(no_undefined_flag, $1)='$wl-z,text' _LT_TAGVAR(allow_undefined_flag, $1)='$wl-z,nodefs' _LT_TAGVAR(archive_cmds_need_lc, $1)=no _LT_TAGVAR(hardcode_shlibpath_var, $1)=no _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-R,$libdir' _LT_TAGVAR(hardcode_libdir_separator, $1)=':' _LT_TAGVAR(link_all_deplibs, $1)=yes _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-Bexport' runpath_var='LD_RUN_PATH' if test yes = "$GCC"; then _LT_TAGVAR(archive_cmds, $1)='$CC -shared $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' else _LT_TAGVAR(archive_cmds, $1)='$CC -G $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -G $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' fi ;; uts4*) _LT_TAGVAR(archive_cmds, $1)='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' _LT_TAGVAR(hardcode_shlibpath_var, $1)=no ;; *) _LT_TAGVAR(ld_shlibs, $1)=no ;; esac if test sni = "$host_vendor"; then case $host in sysv4 | sysv4.2uw2* | sysv4.3* | sysv5*) _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-Blargedynsym' ;; esac fi fi ]) AC_MSG_RESULT([$_LT_TAGVAR(ld_shlibs, $1)]) test no = "$_LT_TAGVAR(ld_shlibs, $1)" && can_build_shared=no _LT_TAGVAR(with_gnu_ld, $1)=$with_gnu_ld _LT_DECL([], [libext], [0], [Old archive suffix (normally "a")])dnl _LT_DECL([], [shrext_cmds], [1], [Shared library suffix (normally ".so")])dnl _LT_DECL([], [extract_expsyms_cmds], [2], [The commands to extract the exported symbol list from a shared archive]) # # Do we need to explicitly link libc? # case "x$_LT_TAGVAR(archive_cmds_need_lc, $1)" in x|xyes) # Assume -lc should be added _LT_TAGVAR(archive_cmds_need_lc, $1)=yes if test yes,yes = "$GCC,$enable_shared"; then case $_LT_TAGVAR(archive_cmds, $1) in *'~'*) # FIXME: we may have to deal with multi-command sequences. ;; '$CC '*) # Test whether the compiler implicitly links with -lc since on some # systems, -lgcc has to come before -lc. If gcc already passes -lc # to ld, don't add -lc before -lgcc. AC_CACHE_CHECK([whether -lc should be explicitly linked in], [lt_cv_]_LT_TAGVAR(archive_cmds_need_lc, $1), [$RM conftest* echo "$lt_simple_compile_test_code" > conftest.$ac_ext if AC_TRY_EVAL(ac_compile) 2>conftest.err; then soname=conftest lib=conftest libobjs=conftest.$ac_objext deplibs= wl=$_LT_TAGVAR(lt_prog_compiler_wl, $1) pic_flag=$_LT_TAGVAR(lt_prog_compiler_pic, $1) compiler_flags=-v linker_flags=-v verstring= output_objdir=. libname=conftest lt_save_allow_undefined_flag=$_LT_TAGVAR(allow_undefined_flag, $1) _LT_TAGVAR(allow_undefined_flag, $1)= if AC_TRY_EVAL(_LT_TAGVAR(archive_cmds, $1) 2\>\&1 \| $GREP \" -lc \" \>/dev/null 2\>\&1) then lt_cv_[]_LT_TAGVAR(archive_cmds_need_lc, $1)=no else lt_cv_[]_LT_TAGVAR(archive_cmds_need_lc, $1)=yes fi _LT_TAGVAR(allow_undefined_flag, $1)=$lt_save_allow_undefined_flag else cat conftest.err 1>&5 fi $RM conftest* ]) _LT_TAGVAR(archive_cmds_need_lc, $1)=$lt_cv_[]_LT_TAGVAR(archive_cmds_need_lc, $1) ;; esac fi ;; esac _LT_TAGDECL([build_libtool_need_lc], [archive_cmds_need_lc], [0], [Whether or not to add -lc for building shared libraries]) _LT_TAGDECL([allow_libtool_libs_with_static_runtimes], [enable_shared_with_static_runtimes], [0], [Whether or not to disallow shared libs when runtime libs are static]) _LT_TAGDECL([], [export_dynamic_flag_spec], [1], [Compiler flag to allow reflexive dlopens]) _LT_TAGDECL([], [whole_archive_flag_spec], [1], [Compiler flag to generate shared objects directly from archives]) _LT_TAGDECL([], [compiler_needs_object], [1], [Whether the compiler copes with passing no objects directly]) _LT_TAGDECL([], [old_archive_from_new_cmds], [2], [Create an old-style archive from a shared archive]) _LT_TAGDECL([], [old_archive_from_expsyms_cmds], [2], [Create a temporary old-style archive to link instead of a shared archive]) _LT_TAGDECL([], [archive_cmds], [2], [Commands used to build a shared archive]) _LT_TAGDECL([], [archive_expsym_cmds], [2]) _LT_TAGDECL([], [module_cmds], [2], [Commands used to build a loadable module if different from building a shared archive.]) _LT_TAGDECL([], [module_expsym_cmds], [2]) _LT_TAGDECL([], [with_gnu_ld], [1], [Whether we are building with GNU ld or not]) _LT_TAGDECL([], [allow_undefined_flag], [1], [Flag that allows shared libraries with undefined symbols to be built]) _LT_TAGDECL([], [no_undefined_flag], [1], [Flag that enforces no undefined symbols]) _LT_TAGDECL([], [hardcode_libdir_flag_spec], [1], [Flag to hardcode $libdir into a binary during linking. This must work even if $libdir does not exist]) _LT_TAGDECL([], [hardcode_libdir_separator], [1], [Whether we need a single "-rpath" flag with a separated argument]) _LT_TAGDECL([], [hardcode_direct], [0], [Set to "yes" if using DIR/libNAME$shared_ext during linking hardcodes DIR into the resulting binary]) _LT_TAGDECL([], [hardcode_direct_absolute], [0], [Set to "yes" if using DIR/libNAME$shared_ext during linking hardcodes DIR into the resulting binary and the resulting library dependency is "absolute", i.e impossible to change by setting $shlibpath_var if the library is relocated]) _LT_TAGDECL([], [hardcode_minus_L], [0], [Set to "yes" if using the -LDIR flag during linking hardcodes DIR into the resulting binary]) _LT_TAGDECL([], [hardcode_shlibpath_var], [0], [Set to "yes" if using SHLIBPATH_VAR=DIR during linking hardcodes DIR into the resulting binary]) _LT_TAGDECL([], [hardcode_automatic], [0], [Set to "yes" if building a shared library automatically hardcodes DIR into the library and all subsequent libraries and executables linked against it]) _LT_TAGDECL([], [inherit_rpath], [0], [Set to yes if linker adds runtime paths of dependent libraries to runtime path list]) _LT_TAGDECL([], [link_all_deplibs], [0], [Whether libtool must link a program against all its dependency libraries]) _LT_TAGDECL([], [always_export_symbols], [0], [Set to "yes" if exported symbols are required]) _LT_TAGDECL([], [export_symbols_cmds], [2], [The commands to list exported symbols]) _LT_TAGDECL([], [exclude_expsyms], [1], [Symbols that should not be listed in the preloaded symbols]) _LT_TAGDECL([], [include_expsyms], [1], [Symbols that must always be exported]) _LT_TAGDECL([], [prelink_cmds], [2], [Commands necessary for linking programs (against libraries) with templates]) _LT_TAGDECL([], [postlink_cmds], [2], [Commands necessary for finishing linking programs]) _LT_TAGDECL([], [file_list_spec], [1], [Specify filename containing input files]) dnl FIXME: Not yet implemented dnl _LT_TAGDECL([], [thread_safe_flag_spec], [1], dnl [Compiler flag to generate thread safe objects]) ])# _LT_LINKER_SHLIBS # _LT_LANG_C_CONFIG([TAG]) # ------------------------ # Ensure that the configuration variables for a C compiler are suitably # defined. These variables are subsequently used by _LT_CONFIG to write # the compiler configuration to 'libtool'. m4_defun([_LT_LANG_C_CONFIG], [m4_require([_LT_DECL_EGREP])dnl lt_save_CC=$CC AC_LANG_PUSH(C) # Source file extension for C test sources. ac_ext=c # Object file extension for compiled C test sources. objext=o _LT_TAGVAR(objext, $1)=$objext # Code to be used in simple compile tests lt_simple_compile_test_code="int some_variable = 0;" # Code to be used in simple link tests lt_simple_link_test_code='int main(){return(0);}' _LT_TAG_COMPILER # Save the default compiler, since it gets overwritten when the other # tags are being tested, and _LT_TAGVAR(compiler, []) is a NOP. compiler_DEFAULT=$CC # save warnings/boilerplate of simple test code _LT_COMPILER_BOILERPLATE _LT_LINKER_BOILERPLATE ## CAVEAT EMPTOR: ## There is no encapsulation within the following macros, do not change ## the running order or otherwise move them around unless you know exactly ## what you are doing... if test -n "$compiler"; then _LT_COMPILER_NO_RTTI($1) _LT_COMPILER_PIC($1) _LT_COMPILER_C_O($1) _LT_COMPILER_FILE_LOCKS($1) _LT_LINKER_SHLIBS($1) _LT_SYS_DYNAMIC_LINKER($1) _LT_LINKER_HARDCODE_LIBPATH($1) LT_SYS_DLOPEN_SELF _LT_CMD_STRIPLIB # Report what library types will actually be built AC_MSG_CHECKING([if libtool supports shared libraries]) AC_MSG_RESULT([$can_build_shared]) AC_MSG_CHECKING([whether to build shared libraries]) test no = "$can_build_shared" && enable_shared=no # On AIX, shared libraries and static libraries use the same namespace, and # are all built from PIC. case $host_os in aix3*) test yes = "$enable_shared" && enable_static=no if test -n "$RANLIB"; then archive_cmds="$archive_cmds~\$RANLIB \$lib" postinstall_cmds='$RANLIB $lib' fi ;; aix[[4-9]]*) if test ia64 != "$host_cpu"; then case $enable_shared,$with_aix_soname,$aix_use_runtimelinking in yes,aix,yes) ;; # shared object as lib.so file only yes,svr4,*) ;; # shared object as lib.so archive member only yes,*) enable_static=no ;; # shared object in lib.a archive as well esac fi ;; esac AC_MSG_RESULT([$enable_shared]) AC_MSG_CHECKING([whether to build static libraries]) # Make sure either enable_shared or enable_static is yes. test yes = "$enable_shared" || enable_static=yes AC_MSG_RESULT([$enable_static]) _LT_CONFIG($1) fi AC_LANG_POP CC=$lt_save_CC ])# _LT_LANG_C_CONFIG # _LT_LANG_CXX_CONFIG([TAG]) # -------------------------- # Ensure that the configuration variables for a C++ compiler are suitably # defined. These variables are subsequently used by _LT_CONFIG to write # the compiler configuration to 'libtool'. m4_defun([_LT_LANG_CXX_CONFIG], [m4_require([_LT_FILEUTILS_DEFAULTS])dnl m4_require([_LT_DECL_EGREP])dnl m4_require([_LT_PATH_MANIFEST_TOOL])dnl if test -n "$CXX" && ( test no != "$CXX" && ( (test g++ = "$CXX" && `g++ -v >/dev/null 2>&1` ) || (test g++ != "$CXX"))); then AC_PROG_CXXCPP else _lt_caught_CXX_error=yes fi AC_LANG_PUSH(C++) _LT_TAGVAR(archive_cmds_need_lc, $1)=no _LT_TAGVAR(allow_undefined_flag, $1)= _LT_TAGVAR(always_export_symbols, $1)=no _LT_TAGVAR(archive_expsym_cmds, $1)= _LT_TAGVAR(compiler_needs_object, $1)=no _LT_TAGVAR(export_dynamic_flag_spec, $1)= _LT_TAGVAR(hardcode_direct, $1)=no _LT_TAGVAR(hardcode_direct_absolute, $1)=no _LT_TAGVAR(hardcode_libdir_flag_spec, $1)= _LT_TAGVAR(hardcode_libdir_separator, $1)= _LT_TAGVAR(hardcode_minus_L, $1)=no _LT_TAGVAR(hardcode_shlibpath_var, $1)=unsupported _LT_TAGVAR(hardcode_automatic, $1)=no _LT_TAGVAR(inherit_rpath, $1)=no _LT_TAGVAR(module_cmds, $1)= _LT_TAGVAR(module_expsym_cmds, $1)= _LT_TAGVAR(link_all_deplibs, $1)=unknown _LT_TAGVAR(old_archive_cmds, $1)=$old_archive_cmds _LT_TAGVAR(reload_flag, $1)=$reload_flag _LT_TAGVAR(reload_cmds, $1)=$reload_cmds _LT_TAGVAR(no_undefined_flag, $1)= _LT_TAGVAR(whole_archive_flag_spec, $1)= _LT_TAGVAR(enable_shared_with_static_runtimes, $1)=no # Source file extension for C++ test sources. ac_ext=cpp # Object file extension for compiled C++ test sources. objext=o _LT_TAGVAR(objext, $1)=$objext # No sense in running all these tests if we already determined that # the CXX compiler isn't working. Some variables (like enable_shared) # are currently assumed to apply to all compilers on this platform, # and will be corrupted by setting them based on a non-working compiler. if test yes != "$_lt_caught_CXX_error"; then # Code to be used in simple compile tests lt_simple_compile_test_code="int some_variable = 0;" # Code to be used in simple link tests lt_simple_link_test_code='int main(int, char *[[]]) { return(0); }' # ltmain only uses $CC for tagged configurations so make sure $CC is set. _LT_TAG_COMPILER # save warnings/boilerplate of simple test code _LT_COMPILER_BOILERPLATE _LT_LINKER_BOILERPLATE # Allow CC to be a program name with arguments. lt_save_CC=$CC lt_save_CFLAGS=$CFLAGS lt_save_LD=$LD lt_save_GCC=$GCC GCC=$GXX lt_save_with_gnu_ld=$with_gnu_ld lt_save_path_LD=$lt_cv_path_LD if test -n "${lt_cv_prog_gnu_ldcxx+set}"; then lt_cv_prog_gnu_ld=$lt_cv_prog_gnu_ldcxx else $as_unset lt_cv_prog_gnu_ld fi if test -n "${lt_cv_path_LDCXX+set}"; then lt_cv_path_LD=$lt_cv_path_LDCXX else $as_unset lt_cv_path_LD fi test -z "${LDCXX+set}" || LD=$LDCXX CC=${CXX-"c++"} CFLAGS=$CXXFLAGS compiler=$CC _LT_TAGVAR(compiler, $1)=$CC _LT_CC_BASENAME([$compiler]) if test -n "$compiler"; then # We don't want -fno-exception when compiling C++ code, so set the # no_builtin_flag separately if test yes = "$GXX"; then _LT_TAGVAR(lt_prog_compiler_no_builtin_flag, $1)=' -fno-builtin' else _LT_TAGVAR(lt_prog_compiler_no_builtin_flag, $1)= fi if test yes = "$GXX"; then # Set up default GNU C++ configuration LT_PATH_LD # Check if GNU C++ uses GNU ld as the underlying linker, since the # archiving commands below assume that GNU ld is being used. if test yes = "$with_gnu_ld"; then _LT_TAGVAR(archive_cmds, $1)='$CC $pic_flag -shared -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname -o $lib' _LT_TAGVAR(archive_expsym_cmds, $1)='$CC $pic_flag -shared -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl--export-dynamic' # If archive_cmds runs LD, not CC, wlarc should be empty # XXX I think wlarc can be eliminated in ltcf-cxx, but I need to # investigate it a little bit more. (MM) wlarc='$wl' # ancient GNU ld didn't support --whole-archive et. al. if eval "`$CC -print-prog-name=ld` --help 2>&1" | $GREP 'no-whole-archive' > /dev/null; then _LT_TAGVAR(whole_archive_flag_spec, $1)=$wlarc'--whole-archive$convenience '$wlarc'--no-whole-archive' else _LT_TAGVAR(whole_archive_flag_spec, $1)= fi else with_gnu_ld=no wlarc= # A generic and very simple default shared library creation # command for GNU C++ for the case where it uses the native # linker, instead of GNU ld. If possible, this setting should # overridden to take advantage of the native linker features on # the platform it is being used on. _LT_TAGVAR(archive_cmds, $1)='$CC -shared -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags -o $lib' fi # Commands to make compiler produce verbose output that lists # what "hidden" libraries, object files and flags are used when # linking a shared library. output_verbose_link_cmd='$CC -shared $CFLAGS -v conftest.$objext 2>&1 | $GREP -v "^Configured with:" | $GREP "\-L"' else GXX=no with_gnu_ld=no wlarc= fi # PORTME: fill in a description of your system's C++ link characteristics AC_MSG_CHECKING([whether the $compiler linker ($LD) supports shared libraries]) _LT_TAGVAR(ld_shlibs, $1)=yes case $host_os in aix3*) # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no ;; aix[[4-9]]*) if test ia64 = "$host_cpu"; then # On IA64, the linker does run time linking by default, so we don't # have to do anything special. aix_use_runtimelinking=no exp_sym_flag='-Bexport' no_entry_flag= else aix_use_runtimelinking=no # Test if we are trying to use run time linking or normal # AIX style linking. If -brtl is somewhere in LDFLAGS, we # have runtime linking enabled, and use it for executables. # For shared libraries, we enable/disable runtime linking # depending on the kind of the shared library created - # when "with_aix_soname,aix_use_runtimelinking" is: # "aix,no" lib.a(lib.so.V) shared, rtl:no, for executables # "aix,yes" lib.so shared, rtl:yes, for executables # lib.a static archive # "both,no" lib.so.V(shr.o) shared, rtl:yes # lib.a(lib.so.V) shared, rtl:no, for executables # "both,yes" lib.so.V(shr.o) shared, rtl:yes, for executables # lib.a(lib.so.V) shared, rtl:no # "svr4,*" lib.so.V(shr.o) shared, rtl:yes, for executables # lib.a static archive case $host_os in aix4.[[23]]|aix4.[[23]].*|aix[[5-9]]*) for ld_flag in $LDFLAGS; do case $ld_flag in *-brtl*) aix_use_runtimelinking=yes break ;; esac done if test svr4,no = "$with_aix_soname,$aix_use_runtimelinking"; then # With aix-soname=svr4, we create the lib.so.V shared archives only, # so we don't have lib.a shared libs to link our executables. # We have to force runtime linking in this case. aix_use_runtimelinking=yes LDFLAGS="$LDFLAGS -Wl,-brtl" fi ;; esac exp_sym_flag='-bexport' no_entry_flag='-bnoentry' fi # When large executables or shared objects are built, AIX ld can # have problems creating the table of contents. If linking a library # or program results in "error TOC overflow" add -mminimal-toc to # CXXFLAGS/CFLAGS for g++/gcc. In the cases where that is not # enough to fix the problem, add -Wl,-bbigtoc to LDFLAGS. _LT_TAGVAR(archive_cmds, $1)='' _LT_TAGVAR(hardcode_direct, $1)=yes _LT_TAGVAR(hardcode_direct_absolute, $1)=yes _LT_TAGVAR(hardcode_libdir_separator, $1)=':' _LT_TAGVAR(link_all_deplibs, $1)=yes _LT_TAGVAR(file_list_spec, $1)='$wl-f,' case $with_aix_soname,$aix_use_runtimelinking in aix,*) ;; # no import file svr4,* | *,yes) # use import file # The Import File defines what to hardcode. _LT_TAGVAR(hardcode_direct, $1)=no _LT_TAGVAR(hardcode_direct_absolute, $1)=no ;; esac if test yes = "$GXX"; then case $host_os in aix4.[[012]]|aix4.[[012]].*) # We only want to do this on AIX 4.2 and lower, the check # below for broken collect2 doesn't work under 4.3+ collect2name=`$CC -print-prog-name=collect2` if test -f "$collect2name" && strings "$collect2name" | $GREP resolve_lib_name >/dev/null then # We have reworked collect2 : else # We have old collect2 _LT_TAGVAR(hardcode_direct, $1)=unsupported # It fails to find uninstalled libraries when the uninstalled # path is not listed in the libpath. Setting hardcode_minus_L # to unsupported forces relinking _LT_TAGVAR(hardcode_minus_L, $1)=yes _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' _LT_TAGVAR(hardcode_libdir_separator, $1)= fi esac shared_flag='-shared' if test yes = "$aix_use_runtimelinking"; then shared_flag=$shared_flag' $wl-G' fi # Need to ensure runtime linking is disabled for the traditional # shared library, or the linker may eventually find shared libraries # /with/ Import File - we do not want to mix them. shared_flag_aix='-shared' shared_flag_svr4='-shared $wl-G' else # not using gcc if test ia64 = "$host_cpu"; then # VisualAge C++, Version 5.5 for AIX 5L for IA-64, Beta 3 Release # chokes on -Wl,-G. The following line is correct: shared_flag='-G' else if test yes = "$aix_use_runtimelinking"; then shared_flag='$wl-G' else shared_flag='$wl-bM:SRE' fi shared_flag_aix='$wl-bM:SRE' shared_flag_svr4='$wl-G' fi fi _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-bexpall' # It seems that -bexpall does not export symbols beginning with # underscore (_), so it is better to generate a list of symbols to # export. _LT_TAGVAR(always_export_symbols, $1)=yes if test aix,yes = "$with_aix_soname,$aix_use_runtimelinking"; then # Warning - without using the other runtime loading flags (-brtl), # -berok will link without error, but may produce a broken library. # The "-G" linker flag allows undefined symbols. _LT_TAGVAR(no_undefined_flag, $1)='-bernotok' # Determine the default libpath from the value encoded in an empty # executable. _LT_SYS_MODULE_PATH_AIX([$1]) _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-blibpath:$libdir:'"$aix_libpath" _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -o $output_objdir/$soname $libobjs $deplibs $wl'$no_entry_flag' $compiler_flags `if test -n "$allow_undefined_flag"; then func_echo_all "$wl$allow_undefined_flag"; else :; fi` $wl'$exp_sym_flag:\$export_symbols' '$shared_flag else if test ia64 = "$host_cpu"; then _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-R $libdir:/usr/lib:/lib' _LT_TAGVAR(allow_undefined_flag, $1)="-z nodefs" _LT_TAGVAR(archive_expsym_cmds, $1)="\$CC $shared_flag"' -o $output_objdir/$soname $libobjs $deplibs '"\$wl$no_entry_flag"' $compiler_flags $wl$allow_undefined_flag '"\$wl$exp_sym_flag:\$export_symbols" else # Determine the default libpath from the value encoded in an # empty executable. _LT_SYS_MODULE_PATH_AIX([$1]) _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-blibpath:$libdir:'"$aix_libpath" # Warning - without using the other run time loading flags, # -berok will link without error, but may produce a broken library. _LT_TAGVAR(no_undefined_flag, $1)=' $wl-bernotok' _LT_TAGVAR(allow_undefined_flag, $1)=' $wl-berok' if test yes = "$with_gnu_ld"; then # We only use this code for GNU lds that support --whole-archive. _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl--whole-archive$convenience $wl--no-whole-archive' else # Exported symbols can be pulled into shared objects from archives _LT_TAGVAR(whole_archive_flag_spec, $1)='$convenience' fi _LT_TAGVAR(archive_cmds_need_lc, $1)=yes _LT_TAGVAR(archive_expsym_cmds, $1)='$RM -r $output_objdir/$realname.d~$MKDIR $output_objdir/$realname.d' # -brtl affects multiple linker settings, -berok does not and is overridden later compiler_flags_filtered='`func_echo_all "$compiler_flags " | $SED -e "s%-brtl\\([[, ]]\\)%-berok\\1%g"`' if test svr4 != "$with_aix_soname"; then # This is similar to how AIX traditionally builds its shared # libraries. Need -bnortl late, we may have -brtl in LDFLAGS. _LT_TAGVAR(archive_expsym_cmds, $1)="$_LT_TAGVAR(archive_expsym_cmds, $1)"'~$CC '$shared_flag_aix' -o $output_objdir/$realname.d/$soname $libobjs $deplibs $wl-bnoentry '$compiler_flags_filtered'$wl-bE:$export_symbols$allow_undefined_flag~$AR $AR_FLAGS $output_objdir/$libname$release.a $output_objdir/$realname.d/$soname' fi if test aix != "$with_aix_soname"; then _LT_TAGVAR(archive_expsym_cmds, $1)="$_LT_TAGVAR(archive_expsym_cmds, $1)"'~$CC '$shared_flag_svr4' -o $output_objdir/$realname.d/$shared_archive_member_spec.o $libobjs $deplibs $wl-bnoentry '$compiler_flags_filtered'$wl-bE:$export_symbols$allow_undefined_flag~$STRIP -e $output_objdir/$realname.d/$shared_archive_member_spec.o~( func_echo_all "#! $soname($shared_archive_member_spec.o)"; if test shr_64 = "$shared_archive_member_spec"; then func_echo_all "# 64"; else func_echo_all "# 32"; fi; cat $export_symbols ) > $output_objdir/$realname.d/$shared_archive_member_spec.imp~$AR $AR_FLAGS $output_objdir/$soname $output_objdir/$realname.d/$shared_archive_member_spec.o $output_objdir/$realname.d/$shared_archive_member_spec.imp' else # used by -dlpreopen to get the symbols _LT_TAGVAR(archive_expsym_cmds, $1)="$_LT_TAGVAR(archive_expsym_cmds, $1)"'~$MV $output_objdir/$realname.d/$soname $output_objdir' fi _LT_TAGVAR(archive_expsym_cmds, $1)="$_LT_TAGVAR(archive_expsym_cmds, $1)"'~$RM -r $output_objdir/$realname.d' fi fi ;; beos*) if $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then _LT_TAGVAR(allow_undefined_flag, $1)=unsupported # Joseph Beckenbach says some releases of gcc # support --undefined. This deserves some investigation. FIXME _LT_TAGVAR(archive_cmds, $1)='$CC -nostart $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' else _LT_TAGVAR(ld_shlibs, $1)=no fi ;; chorus*) case $cc_basename in *) # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no ;; esac ;; cygwin* | mingw* | pw32* | cegcc*) case $GXX,$cc_basename in ,cl* | no,cl*) # Native MSVC # hardcode_libdir_flag_spec is actually meaningless, as there is # no search path for DLLs. _LT_TAGVAR(hardcode_libdir_flag_spec, $1)=' ' _LT_TAGVAR(allow_undefined_flag, $1)=unsupported _LT_TAGVAR(always_export_symbols, $1)=yes _LT_TAGVAR(file_list_spec, $1)='@' # Tell ltmain to make .lib files, not .a files. libext=lib # Tell ltmain to make .dll files, not .so files. shrext_cmds=.dll # FIXME: Setting linknames here is a bad hack. _LT_TAGVAR(archive_cmds, $1)='$CC -o $output_objdir/$soname $libobjs $compiler_flags $deplibs -Wl,-DLL,-IMPLIB:"$tool_output_objdir$libname.dll.lib"~linknames=' _LT_TAGVAR(archive_expsym_cmds, $1)='if _LT_DLL_DEF_P([$export_symbols]); then cp "$export_symbols" "$output_objdir/$soname.def"; echo "$tool_output_objdir$soname.def" > "$output_objdir/$soname.exp"; else $SED -e '\''s/^/-link -EXPORT:/'\'' < $export_symbols > $output_objdir/$soname.exp; fi~ $CC -o $tool_output_objdir$soname $libobjs $compiler_flags $deplibs "@$tool_output_objdir$soname.exp" -Wl,-DLL,-IMPLIB:"$tool_output_objdir$libname.dll.lib"~ linknames=' # The linker will not automatically build a static lib if we build a DLL. # _LT_TAGVAR(old_archive_from_new_cmds, $1)='true' _LT_TAGVAR(enable_shared_with_static_runtimes, $1)=yes # Don't use ranlib _LT_TAGVAR(old_postinstall_cmds, $1)='chmod 644 $oldlib' _LT_TAGVAR(postlink_cmds, $1)='lt_outputfile="@OUTPUT@"~ lt_tool_outputfile="@TOOL_OUTPUT@"~ case $lt_outputfile in *.exe|*.EXE) ;; *) lt_outputfile=$lt_outputfile.exe lt_tool_outputfile=$lt_tool_outputfile.exe ;; esac~ func_to_tool_file "$lt_outputfile"~ if test : != "$MANIFEST_TOOL" && test -f "$lt_outputfile.manifest"; then $MANIFEST_TOOL -manifest "$lt_tool_outputfile.manifest" -outputresource:"$lt_tool_outputfile" || exit 1; $RM "$lt_outputfile.manifest"; fi' ;; *) # g++ # _LT_TAGVAR(hardcode_libdir_flag_spec, $1) is actually meaningless, # as there is no search path for DLLs. _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl--export-all-symbols' _LT_TAGVAR(allow_undefined_flag, $1)=unsupported _LT_TAGVAR(always_export_symbols, $1)=no _LT_TAGVAR(enable_shared_with_static_runtimes, $1)=yes if $LD --help 2>&1 | $GREP 'auto-import' > /dev/null; then _LT_TAGVAR(archive_cmds, $1)='$CC -shared -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags -o $output_objdir/$soname $wl--enable-auto-image-base -Xlinker --out-implib -Xlinker $lib' # If the export-symbols file already is a .def file, use it as # is; otherwise, prepend EXPORTS... _LT_TAGVAR(archive_expsym_cmds, $1)='if _LT_DLL_DEF_P([$export_symbols]); then cp $export_symbols $output_objdir/$soname.def; else echo EXPORTS > $output_objdir/$soname.def; cat $export_symbols >> $output_objdir/$soname.def; fi~ $CC -shared -nostdlib $output_objdir/$soname.def $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags -o $output_objdir/$soname $wl--enable-auto-image-base -Xlinker --out-implib -Xlinker $lib' else _LT_TAGVAR(ld_shlibs, $1)=no fi ;; esac ;; darwin* | rhapsody*) _LT_DARWIN_LINKER_FEATURES($1) ;; os2*) _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' _LT_TAGVAR(hardcode_minus_L, $1)=yes _LT_TAGVAR(allow_undefined_flag, $1)=unsupported shrext_cmds=.dll _LT_TAGVAR(archive_cmds, $1)='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ $ECHO EXPORTS >> $output_objdir/$libname.def~ emxexp $libobjs | $SED /"_DLL_InitTerm"/d >> $output_objdir/$libname.def~ $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ emximp -o $lib $output_objdir/$libname.def' _LT_TAGVAR(archive_expsym_cmds, $1)='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ $ECHO EXPORTS >> $output_objdir/$libname.def~ prefix_cmds="$SED"~ if test EXPORTS = "`$SED 1q $export_symbols`"; then prefix_cmds="$prefix_cmds -e 1d"; fi~ prefix_cmds="$prefix_cmds -e \"s/^\(.*\)$/_\1/g\""~ cat $export_symbols | $prefix_cmds >> $output_objdir/$libname.def~ $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ emximp -o $lib $output_objdir/$libname.def' _LT_TAGVAR(old_archive_From_new_cmds, $1)='emximp -o $output_objdir/${libname}_dll.a $output_objdir/$libname.def' _LT_TAGVAR(enable_shared_with_static_runtimes, $1)=yes ;; dgux*) case $cc_basename in ec++*) # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no ;; ghcx*) # Green Hills C++ Compiler # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no ;; *) # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no ;; esac ;; freebsd2.*) # C++ shared libraries reported to be fairly broken before # switch to ELF _LT_TAGVAR(ld_shlibs, $1)=no ;; freebsd-elf*) _LT_TAGVAR(archive_cmds_need_lc, $1)=no ;; freebsd* | dragonfly*) # FreeBSD 3 and later use GNU C++ and GNU ld with standard ELF # conventions _LT_TAGVAR(ld_shlibs, $1)=yes ;; haiku*) _LT_TAGVAR(archive_cmds, $1)='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' _LT_TAGVAR(link_all_deplibs, $1)=yes ;; hpux9*) _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl+b $wl$libdir' _LT_TAGVAR(hardcode_libdir_separator, $1)=: _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-E' _LT_TAGVAR(hardcode_direct, $1)=yes _LT_TAGVAR(hardcode_minus_L, $1)=yes # Not in the search PATH, # but as the default # location of the library. case $cc_basename in CC*) # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no ;; aCC*) _LT_TAGVAR(archive_cmds, $1)='$RM $output_objdir/$soname~$CC -b $wl+b $wl$install_libdir -o $output_objdir/$soname $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags~test "x$output_objdir/$soname" = "x$lib" || mv $output_objdir/$soname $lib' # Commands to make compiler produce verbose output that lists # what "hidden" libraries, object files and flags are used when # linking a shared library. # # There doesn't appear to be a way to prevent this compiler from # explicitly linking system object files so we need to strip them # from the output so that they don't get included in the library # dependencies. output_verbose_link_cmd='templist=`($CC -b $CFLAGS -v conftest.$objext 2>&1) | $EGREP "\-L"`; list= ; for z in $templist; do case $z in conftest.$objext) list="$list $z";; *.$objext);; *) list="$list $z";;esac; done; func_echo_all "$list"' ;; *) if test yes = "$GXX"; then _LT_TAGVAR(archive_cmds, $1)='$RM $output_objdir/$soname~$CC -shared -nostdlib $pic_flag $wl+b $wl$install_libdir -o $output_objdir/$soname $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags~test "x$output_objdir/$soname" = "x$lib" || mv $output_objdir/$soname $lib' else # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no fi ;; esac ;; hpux10*|hpux11*) if test no = "$with_gnu_ld"; then _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl+b $wl$libdir' _LT_TAGVAR(hardcode_libdir_separator, $1)=: case $host_cpu in hppa*64*|ia64*) ;; *) _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-E' ;; esac fi case $host_cpu in hppa*64*|ia64*) _LT_TAGVAR(hardcode_direct, $1)=no _LT_TAGVAR(hardcode_shlibpath_var, $1)=no ;; *) _LT_TAGVAR(hardcode_direct, $1)=yes _LT_TAGVAR(hardcode_direct_absolute, $1)=yes _LT_TAGVAR(hardcode_minus_L, $1)=yes # Not in the search PATH, # but as the default # location of the library. ;; esac case $cc_basename in CC*) # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no ;; aCC*) case $host_cpu in hppa*64*) _LT_TAGVAR(archive_cmds, $1)='$CC -b $wl+h $wl$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' ;; ia64*) _LT_TAGVAR(archive_cmds, $1)='$CC -b $wl+h $wl$soname $wl+nodefaultrpath -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' ;; *) _LT_TAGVAR(archive_cmds, $1)='$CC -b $wl+h $wl$soname $wl+b $wl$install_libdir -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' ;; esac # Commands to make compiler produce verbose output that lists # what "hidden" libraries, object files and flags are used when # linking a shared library. # # There doesn't appear to be a way to prevent this compiler from # explicitly linking system object files so we need to strip them # from the output so that they don't get included in the library # dependencies. output_verbose_link_cmd='templist=`($CC -b $CFLAGS -v conftest.$objext 2>&1) | $GREP "\-L"`; list= ; for z in $templist; do case $z in conftest.$objext) list="$list $z";; *.$objext);; *) list="$list $z";;esac; done; func_echo_all "$list"' ;; *) if test yes = "$GXX"; then if test no = "$with_gnu_ld"; then case $host_cpu in hppa*64*) _LT_TAGVAR(archive_cmds, $1)='$CC -shared -nostdlib -fPIC $wl+h $wl$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' ;; ia64*) _LT_TAGVAR(archive_cmds, $1)='$CC -shared -nostdlib $pic_flag $wl+h $wl$soname $wl+nodefaultrpath -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' ;; *) _LT_TAGVAR(archive_cmds, $1)='$CC -shared -nostdlib $pic_flag $wl+h $wl$soname $wl+b $wl$install_libdir -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' ;; esac fi else # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no fi ;; esac ;; interix[[3-9]]*) _LT_TAGVAR(hardcode_direct, $1)=no _LT_TAGVAR(hardcode_shlibpath_var, $1)=no _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath,$libdir' _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-E' # Hack: On Interix 3.x, we cannot compile PIC because of a broken gcc. # Instead, shared libraries are loaded at an image base (0x10000000 by # default) and relocated if they conflict, which is a slow very memory # consuming and fragmenting process. To avoid this, we pick a random, # 256 KiB-aligned image base between 0x50000000 and 0x6FFC0000 at link # time. Moving up from 0x10000000 also allows more sbrk(2) space. _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-h,$soname $wl--image-base,`expr ${RANDOM-$$} % 4096 / 2 \* 262144 + 1342177280` -o $lib' _LT_TAGVAR(archive_expsym_cmds, $1)='sed "s|^|_|" $export_symbols >$output_objdir/$soname.expsym~$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-h,$soname $wl--retain-symbols-file,$output_objdir/$soname.expsym $wl--image-base,`expr ${RANDOM-$$} % 4096 / 2 \* 262144 + 1342177280` -o $lib' ;; irix5* | irix6*) case $cc_basename in CC*) # SGI C++ _LT_TAGVAR(archive_cmds, $1)='$CC -shared -all -multigot $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib' # Archives containing C++ object files must be created using # "CC -ar", where "CC" is the IRIX C++ compiler. This is # necessary to make sure instantiated templates are included # in the archive. _LT_TAGVAR(old_archive_cmds, $1)='$CC -ar -WR,-u -o $oldlib $oldobjs' ;; *) if test yes = "$GXX"; then if test no = "$with_gnu_ld"; then _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations -o $lib' else _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` -o $lib' fi fi _LT_TAGVAR(link_all_deplibs, $1)=yes ;; esac _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' _LT_TAGVAR(hardcode_libdir_separator, $1)=: _LT_TAGVAR(inherit_rpath, $1)=yes ;; linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*) case $cc_basename in KCC*) # Kuck and Associates, Inc. (KAI) C++ Compiler # KCC will only create a shared library if the output file # ends with ".so" (or ".sl" for HP-UX), so rename the library # to its proper name (with version) after linking. _LT_TAGVAR(archive_cmds, $1)='tempext=`echo $shared_ext | $SED -e '\''s/\([[^()0-9A-Za-z{}]]\)/\\\\\1/g'\''`; templib=`echo $lib | $SED -e "s/\$tempext\..*/.so/"`; $CC $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags --soname $soname -o \$templib; mv \$templib $lib' _LT_TAGVAR(archive_expsym_cmds, $1)='tempext=`echo $shared_ext | $SED -e '\''s/\([[^()0-9A-Za-z{}]]\)/\\\\\1/g'\''`; templib=`echo $lib | $SED -e "s/\$tempext\..*/.so/"`; $CC $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags --soname $soname -o \$templib $wl-retain-symbols-file,$export_symbols; mv \$templib $lib' # Commands to make compiler produce verbose output that lists # what "hidden" libraries, object files and flags are used when # linking a shared library. # # There doesn't appear to be a way to prevent this compiler from # explicitly linking system object files so we need to strip them # from the output so that they don't get included in the library # dependencies. output_verbose_link_cmd='templist=`$CC $CFLAGS -v conftest.$objext -o libconftest$shared_ext 2>&1 | $GREP "ld"`; rm -f libconftest$shared_ext; list= ; for z in $templist; do case $z in conftest.$objext) list="$list $z";; *.$objext);; *) list="$list $z";;esac; done; func_echo_all "$list"' _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath,$libdir' _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl--export-dynamic' # Archives containing C++ object files must be created using # "CC -Bstatic", where "CC" is the KAI C++ compiler. _LT_TAGVAR(old_archive_cmds, $1)='$CC -Bstatic -o $oldlib $oldobjs' ;; icpc* | ecpc* ) # Intel C++ with_gnu_ld=yes # version 8.0 and above of icpc choke on multiply defined symbols # if we add $predep_objects and $postdep_objects, however 7.1 and # earlier do not add the objects themselves. case `$CC -V 2>&1` in *"Version 7."*) _LT_TAGVAR(archive_cmds, $1)='$CC -shared $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname -o $lib' _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' ;; *) # Version 8.0 or newer tmp_idyn= case $host_cpu in ia64*) tmp_idyn=' -i_dynamic';; esac _LT_TAGVAR(archive_cmds, $1)='$CC -shared'"$tmp_idyn"' $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared'"$tmp_idyn"' $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' ;; esac _LT_TAGVAR(archive_cmds_need_lc, $1)=no _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath,$libdir' _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl--export-dynamic' _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl--whole-archive$convenience $wl--no-whole-archive' ;; pgCC* | pgcpp*) # Portland Group C++ compiler case `$CC -V` in *pgCC\ [[1-5]].* | *pgcpp\ [[1-5]].*) _LT_TAGVAR(prelink_cmds, $1)='tpldir=Template.dir~ rm -rf $tpldir~ $CC --prelink_objects --instantiation_dir $tpldir $objs $libobjs $compile_deplibs~ compile_command="$compile_command `find $tpldir -name \*.o | sort | $NL2SP`"' _LT_TAGVAR(old_archive_cmds, $1)='tpldir=Template.dir~ rm -rf $tpldir~ $CC --prelink_objects --instantiation_dir $tpldir $oldobjs$old_deplibs~ $AR $AR_FLAGS $oldlib$oldobjs$old_deplibs `find $tpldir -name \*.o | sort | $NL2SP`~ $RANLIB $oldlib' _LT_TAGVAR(archive_cmds, $1)='tpldir=Template.dir~ rm -rf $tpldir~ $CC --prelink_objects --instantiation_dir $tpldir $predep_objects $libobjs $deplibs $convenience $postdep_objects~ $CC -shared $pic_flag $predep_objects $libobjs $deplibs `find $tpldir -name \*.o | sort | $NL2SP` $postdep_objects $compiler_flags $wl-soname $wl$soname -o $lib' _LT_TAGVAR(archive_expsym_cmds, $1)='tpldir=Template.dir~ rm -rf $tpldir~ $CC --prelink_objects --instantiation_dir $tpldir $predep_objects $libobjs $deplibs $convenience $postdep_objects~ $CC -shared $pic_flag $predep_objects $libobjs $deplibs `find $tpldir -name \*.o | sort | $NL2SP` $postdep_objects $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' ;; *) # Version 6 and above use weak symbols _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname -o $lib' _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $pic_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' ;; esac _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl--rpath $wl$libdir' _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl--export-dynamic' _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl--whole-archive`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' ;; cxx*) # Compaq C++ _LT_TAGVAR(archive_cmds, $1)='$CC -shared $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname -o $lib' _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname -o $lib $wl-retain-symbols-file $wl$export_symbols' runpath_var=LD_RUN_PATH _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-rpath $libdir' _LT_TAGVAR(hardcode_libdir_separator, $1)=: # Commands to make compiler produce verbose output that lists # what "hidden" libraries, object files and flags are used when # linking a shared library. # # There doesn't appear to be a way to prevent this compiler from # explicitly linking system object files so we need to strip them # from the output so that they don't get included in the library # dependencies. output_verbose_link_cmd='templist=`$CC -shared $CFLAGS -v conftest.$objext 2>&1 | $GREP "ld"`; templist=`func_echo_all "$templist" | $SED "s/\(^.*ld.*\)\( .*ld .*$\)/\1/"`; list= ; for z in $templist; do case $z in conftest.$objext) list="$list $z";; *.$objext);; *) list="$list $z";;esac; done; func_echo_all "X$list" | $Xsed' ;; xl* | mpixl* | bgxl*) # IBM XL 8.0 on PPC, with GNU ld _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl--export-dynamic' _LT_TAGVAR(archive_cmds, $1)='$CC -qmkshrobj $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' if test yes = "$supports_anon_versioning"; then _LT_TAGVAR(archive_expsym_cmds, $1)='echo "{ global:" > $output_objdir/$libname.ver~ cat $export_symbols | sed -e "s/\(.*\)/\1;/" >> $output_objdir/$libname.ver~ echo "local: *; };" >> $output_objdir/$libname.ver~ $CC -qmkshrobj $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-version-script $wl$output_objdir/$libname.ver -o $lib' fi ;; *) case `$CC -V 2>&1 | sed 5q` in *Sun\ C*) # Sun C++ 5.9 _LT_TAGVAR(no_undefined_flag, $1)=' -zdefs' _LT_TAGVAR(archive_cmds, $1)='$CC -G$allow_undefined_flag -h$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -G$allow_undefined_flag -h$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-retain-symbols-file $wl$export_symbols' _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-R$libdir' _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl--whole-archive`new_convenience=; for conv in $convenience\"\"; do test -z \"$conv\" || new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' _LT_TAGVAR(compiler_needs_object, $1)=yes # Not sure whether something based on # $CC $CFLAGS -v conftest.$objext -o libconftest$shared_ext 2>&1 # would be better. output_verbose_link_cmd='func_echo_all' # Archives containing C++ object files must be created using # "CC -xar", where "CC" is the Sun C++ compiler. This is # necessary to make sure instantiated templates are included # in the archive. _LT_TAGVAR(old_archive_cmds, $1)='$CC -xar -o $oldlib $oldobjs' ;; esac ;; esac ;; lynxos*) # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no ;; m88k*) # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no ;; mvs*) case $cc_basename in cxx*) # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no ;; *) # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no ;; esac ;; netbsd*) if echo __ELF__ | $CC -E - | $GREP __ELF__ >/dev/null; then _LT_TAGVAR(archive_cmds, $1)='$LD -Bshareable -o $lib $predep_objects $libobjs $deplibs $postdep_objects $linker_flags' wlarc= _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-R$libdir' _LT_TAGVAR(hardcode_direct, $1)=yes _LT_TAGVAR(hardcode_shlibpath_var, $1)=no fi # Workaround some broken pre-1.5 toolchains output_verbose_link_cmd='$CC -shared $CFLAGS -v conftest.$objext 2>&1 | $GREP conftest.$objext | $SED -e "s:-lgcc -lc -lgcc::"' ;; *nto* | *qnx*) _LT_TAGVAR(ld_shlibs, $1)=yes ;; openbsd* | bitrig*) if test -f /usr/libexec/ld.so; then _LT_TAGVAR(hardcode_direct, $1)=yes _LT_TAGVAR(hardcode_shlibpath_var, $1)=no _LT_TAGVAR(hardcode_direct_absolute, $1)=yes _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags -o $lib' _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath,$libdir' if test -z "`echo __ELF__ | $CC -E - | grep __ELF__`"; then _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $pic_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-retain-symbols-file,$export_symbols -o $lib' _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-E' _LT_TAGVAR(whole_archive_flag_spec, $1)=$wlarc'--whole-archive$convenience '$wlarc'--no-whole-archive' fi output_verbose_link_cmd=func_echo_all else _LT_TAGVAR(ld_shlibs, $1)=no fi ;; osf3* | osf4* | osf5*) case $cc_basename in KCC*) # Kuck and Associates, Inc. (KAI) C++ Compiler # KCC will only create a shared library if the output file # ends with ".so" (or ".sl" for HP-UX), so rename the library # to its proper name (with version) after linking. _LT_TAGVAR(archive_cmds, $1)='tempext=`echo $shared_ext | $SED -e '\''s/\([[^()0-9A-Za-z{}]]\)/\\\\\1/g'\''`; templib=`echo "$lib" | $SED -e "s/\$tempext\..*/.so/"`; $CC $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags --soname $soname -o \$templib; mv \$templib $lib' _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath,$libdir' _LT_TAGVAR(hardcode_libdir_separator, $1)=: # Archives containing C++ object files must be created using # the KAI C++ compiler. case $host in osf3*) _LT_TAGVAR(old_archive_cmds, $1)='$CC -Bstatic -o $oldlib $oldobjs' ;; *) _LT_TAGVAR(old_archive_cmds, $1)='$CC -o $oldlib $oldobjs' ;; esac ;; RCC*) # Rational C++ 2.4.1 # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no ;; cxx*) case $host in osf3*) _LT_TAGVAR(allow_undefined_flag, $1)=' $wl-expect_unresolved $wl\*' _LT_TAGVAR(archive_cmds, $1)='$CC -shared$allow_undefined_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $soname `test -n "$verstring" && func_echo_all "$wl-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib' _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' ;; *) _LT_TAGVAR(allow_undefined_flag, $1)=' -expect_unresolved \*' _LT_TAGVAR(archive_cmds, $1)='$CC -shared$allow_undefined_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags -msym -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib' _LT_TAGVAR(archive_expsym_cmds, $1)='for i in `cat $export_symbols`; do printf "%s %s\\n" -exported_symbol "\$i" >> $lib.exp; done~ echo "-hidden">> $lib.exp~ $CC -shared$allow_undefined_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags -msym -soname $soname $wl-input $wl$lib.exp `test -n "$verstring" && $ECHO "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib~ $RM $lib.exp' _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-rpath $libdir' ;; esac _LT_TAGVAR(hardcode_libdir_separator, $1)=: # Commands to make compiler produce verbose output that lists # what "hidden" libraries, object files and flags are used when # linking a shared library. # # There doesn't appear to be a way to prevent this compiler from # explicitly linking system object files so we need to strip them # from the output so that they don't get included in the library # dependencies. output_verbose_link_cmd='templist=`$CC -shared $CFLAGS -v conftest.$objext 2>&1 | $GREP "ld" | $GREP -v "ld:"`; templist=`func_echo_all "$templist" | $SED "s/\(^.*ld.*\)\( .*ld.*$\)/\1/"`; list= ; for z in $templist; do case $z in conftest.$objext) list="$list $z";; *.$objext);; *) list="$list $z";;esac; done; func_echo_all "$list"' ;; *) if test yes,no = "$GXX,$with_gnu_ld"; then _LT_TAGVAR(allow_undefined_flag, $1)=' $wl-expect_unresolved $wl\*' case $host in osf3*) _LT_TAGVAR(archive_cmds, $1)='$CC -shared -nostdlib $allow_undefined_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations -o $lib' ;; *) _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag -nostdlib $allow_undefined_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-msym $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations -o $lib' ;; esac _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' _LT_TAGVAR(hardcode_libdir_separator, $1)=: # Commands to make compiler produce verbose output that lists # what "hidden" libraries, object files and flags are used when # linking a shared library. output_verbose_link_cmd='$CC -shared $CFLAGS -v conftest.$objext 2>&1 | $GREP -v "^Configured with:" | $GREP "\-L"' else # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no fi ;; esac ;; psos*) # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no ;; sunos4*) case $cc_basename in CC*) # Sun C++ 4.x # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no ;; lcc*) # Lucid # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no ;; *) # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no ;; esac ;; solaris*) case $cc_basename in CC* | sunCC*) # Sun C++ 4.2, 5.x and Centerline C++ _LT_TAGVAR(archive_cmds_need_lc,$1)=yes _LT_TAGVAR(no_undefined_flag, $1)=' -zdefs' _LT_TAGVAR(archive_cmds, $1)='$CC -G$allow_undefined_flag -h$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' _LT_TAGVAR(archive_expsym_cmds, $1)='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ $CC -G$allow_undefined_flag $wl-M $wl$lib.exp -h$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags~$RM $lib.exp' _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-R$libdir' _LT_TAGVAR(hardcode_shlibpath_var, $1)=no case $host_os in solaris2.[[0-5]] | solaris2.[[0-5]].*) ;; *) # The compiler driver will combine and reorder linker options, # but understands '-z linker_flag'. # Supported since Solaris 2.6 (maybe 2.5.1?) _LT_TAGVAR(whole_archive_flag_spec, $1)='-z allextract$convenience -z defaultextract' ;; esac _LT_TAGVAR(link_all_deplibs, $1)=yes output_verbose_link_cmd='func_echo_all' # Archives containing C++ object files must be created using # "CC -xar", where "CC" is the Sun C++ compiler. This is # necessary to make sure instantiated templates are included # in the archive. _LT_TAGVAR(old_archive_cmds, $1)='$CC -xar -o $oldlib $oldobjs' ;; gcx*) # Green Hills C++ Compiler _LT_TAGVAR(archive_cmds, $1)='$CC -shared $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-h $wl$soname -o $lib' # The C++ compiler must be used to create the archive. _LT_TAGVAR(old_archive_cmds, $1)='$CC $LDFLAGS -archive -o $oldlib $oldobjs' ;; *) # GNU C++ compiler with Solaris linker if test yes,no = "$GXX,$with_gnu_ld"; then _LT_TAGVAR(no_undefined_flag, $1)=' $wl-z ${wl}defs' if $CC --version | $GREP -v '^2\.7' > /dev/null; then _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-h $wl$soname -o $lib' _LT_TAGVAR(archive_expsym_cmds, $1)='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ $CC -shared $pic_flag -nostdlib $wl-M $wl$lib.exp $wl-h $wl$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags~$RM $lib.exp' # Commands to make compiler produce verbose output that lists # what "hidden" libraries, object files and flags are used when # linking a shared library. output_verbose_link_cmd='$CC -shared $CFLAGS -v conftest.$objext 2>&1 | $GREP -v "^Configured with:" | $GREP "\-L"' else # g++ 2.7 appears to require '-G' NOT '-shared' on this # platform. _LT_TAGVAR(archive_cmds, $1)='$CC -G -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-h $wl$soname -o $lib' _LT_TAGVAR(archive_expsym_cmds, $1)='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ $CC -G -nostdlib $wl-M $wl$lib.exp $wl-h $wl$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags~$RM $lib.exp' # Commands to make compiler produce verbose output that lists # what "hidden" libraries, object files and flags are used when # linking a shared library. output_verbose_link_cmd='$CC -G $CFLAGS -v conftest.$objext 2>&1 | $GREP -v "^Configured with:" | $GREP "\-L"' fi _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-R $wl$libdir' case $host_os in solaris2.[[0-5]] | solaris2.[[0-5]].*) ;; *) _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl-z ${wl}allextract$convenience $wl-z ${wl}defaultextract' ;; esac fi ;; esac ;; sysv4*uw2* | sysv5OpenUNIX* | sysv5UnixWare7.[[01]].[[10]]* | unixware7* | sco3.2v5.0.[[024]]*) _LT_TAGVAR(no_undefined_flag, $1)='$wl-z,text' _LT_TAGVAR(archive_cmds_need_lc, $1)=no _LT_TAGVAR(hardcode_shlibpath_var, $1)=no runpath_var='LD_RUN_PATH' case $cc_basename in CC*) _LT_TAGVAR(archive_cmds, $1)='$CC -G $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -G $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' ;; *) _LT_TAGVAR(archive_cmds, $1)='$CC -shared $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' ;; esac ;; sysv5* | sco3.2v5* | sco5v6*) # Note: We CANNOT use -z defs as we might desire, because we do not # link with -lc, and that would cause any symbols used from libc to # always be unresolved, which means just about no library would # ever link correctly. If we're not using GNU ld we use -z text # though, which does catch some bad symbols but isn't as heavy-handed # as -z defs. _LT_TAGVAR(no_undefined_flag, $1)='$wl-z,text' _LT_TAGVAR(allow_undefined_flag, $1)='$wl-z,nodefs' _LT_TAGVAR(archive_cmds_need_lc, $1)=no _LT_TAGVAR(hardcode_shlibpath_var, $1)=no _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-R,$libdir' _LT_TAGVAR(hardcode_libdir_separator, $1)=':' _LT_TAGVAR(link_all_deplibs, $1)=yes _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-Bexport' runpath_var='LD_RUN_PATH' case $cc_basename in CC*) _LT_TAGVAR(archive_cmds, $1)='$CC -G $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -G $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' _LT_TAGVAR(old_archive_cmds, $1)='$CC -Tprelink_objects $oldobjs~ '"$_LT_TAGVAR(old_archive_cmds, $1)" _LT_TAGVAR(reload_cmds, $1)='$CC -Tprelink_objects $reload_objs~ '"$_LT_TAGVAR(reload_cmds, $1)" ;; *) _LT_TAGVAR(archive_cmds, $1)='$CC -shared $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' ;; esac ;; tandem*) case $cc_basename in NCC*) # NonStop-UX NCC 3.20 # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no ;; *) # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no ;; esac ;; vxworks*) # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no ;; *) # FIXME: insert proper C++ library support _LT_TAGVAR(ld_shlibs, $1)=no ;; esac AC_MSG_RESULT([$_LT_TAGVAR(ld_shlibs, $1)]) test no = "$_LT_TAGVAR(ld_shlibs, $1)" && can_build_shared=no _LT_TAGVAR(GCC, $1)=$GXX _LT_TAGVAR(LD, $1)=$LD ## CAVEAT EMPTOR: ## There is no encapsulation within the following macros, do not change ## the running order or otherwise move them around unless you know exactly ## what you are doing... _LT_SYS_HIDDEN_LIBDEPS($1) _LT_COMPILER_PIC($1) _LT_COMPILER_C_O($1) _LT_COMPILER_FILE_LOCKS($1) _LT_LINKER_SHLIBS($1) _LT_SYS_DYNAMIC_LINKER($1) _LT_LINKER_HARDCODE_LIBPATH($1) _LT_CONFIG($1) fi # test -n "$compiler" CC=$lt_save_CC CFLAGS=$lt_save_CFLAGS LDCXX=$LD LD=$lt_save_LD GCC=$lt_save_GCC with_gnu_ld=$lt_save_with_gnu_ld lt_cv_path_LDCXX=$lt_cv_path_LD lt_cv_path_LD=$lt_save_path_LD lt_cv_prog_gnu_ldcxx=$lt_cv_prog_gnu_ld lt_cv_prog_gnu_ld=$lt_save_with_gnu_ld fi # test yes != "$_lt_caught_CXX_error" AC_LANG_POP ])# _LT_LANG_CXX_CONFIG # _LT_FUNC_STRIPNAME_CNF # ---------------------- # func_stripname_cnf prefix suffix name # strip PREFIX and SUFFIX off of NAME. # PREFIX and SUFFIX must not contain globbing or regex special # characters, hashes, percent signs, but SUFFIX may contain a leading # dot (in which case that matches only a dot). # # This function is identical to the (non-XSI) version of func_stripname, # except this one can be used by m4 code that may be executed by configure, # rather than the libtool script. m4_defun([_LT_FUNC_STRIPNAME_CNF],[dnl AC_REQUIRE([_LT_DECL_SED]) AC_REQUIRE([_LT_PROG_ECHO_BACKSLASH]) func_stripname_cnf () { case @S|@2 in .*) func_stripname_result=`$ECHO "@S|@3" | $SED "s%^@S|@1%%; s%\\\\@S|@2\$%%"`;; *) func_stripname_result=`$ECHO "@S|@3" | $SED "s%^@S|@1%%; s%@S|@2\$%%"`;; esac } # func_stripname_cnf ])# _LT_FUNC_STRIPNAME_CNF # _LT_SYS_HIDDEN_LIBDEPS([TAGNAME]) # --------------------------------- # Figure out "hidden" library dependencies from verbose # compiler output when linking a shared library. # Parse the compiler output and extract the necessary # objects, libraries and library flags. m4_defun([_LT_SYS_HIDDEN_LIBDEPS], [m4_require([_LT_FILEUTILS_DEFAULTS])dnl AC_REQUIRE([_LT_FUNC_STRIPNAME_CNF])dnl # Dependencies to place before and after the object being linked: _LT_TAGVAR(predep_objects, $1)= _LT_TAGVAR(postdep_objects, $1)= _LT_TAGVAR(predeps, $1)= _LT_TAGVAR(postdeps, $1)= _LT_TAGVAR(compiler_lib_search_path, $1)= dnl we can't use the lt_simple_compile_test_code here, dnl because it contains code intended for an executable, dnl not a library. It's possible we should let each dnl tag define a new lt_????_link_test_code variable, dnl but it's only used here... m4_if([$1], [], [cat > conftest.$ac_ext <<_LT_EOF int a; void foo (void) { a = 0; } _LT_EOF ], [$1], [CXX], [cat > conftest.$ac_ext <<_LT_EOF class Foo { public: Foo (void) { a = 0; } private: int a; }; _LT_EOF ], [$1], [F77], [cat > conftest.$ac_ext <<_LT_EOF subroutine foo implicit none integer*4 a a=0 return end _LT_EOF ], [$1], [FC], [cat > conftest.$ac_ext <<_LT_EOF subroutine foo implicit none integer a a=0 return end _LT_EOF ], [$1], [GCJ], [cat > conftest.$ac_ext <<_LT_EOF public class foo { private int a; public void bar (void) { a = 0; } }; _LT_EOF ], [$1], [GO], [cat > conftest.$ac_ext <<_LT_EOF package foo func foo() { } _LT_EOF ]) _lt_libdeps_save_CFLAGS=$CFLAGS case "$CC $CFLAGS " in #( *\ -flto*\ *) CFLAGS="$CFLAGS -fno-lto" ;; *\ -fwhopr*\ *) CFLAGS="$CFLAGS -fno-whopr" ;; *\ -fuse-linker-plugin*\ *) CFLAGS="$CFLAGS -fno-use-linker-plugin" ;; esac dnl Parse the compiler output and extract the necessary dnl objects, libraries and library flags. if AC_TRY_EVAL(ac_compile); then # Parse the compiler output and extract the necessary # objects, libraries and library flags. # Sentinel used to keep track of whether or not we are before # the conftest object file. pre_test_object_deps_done=no for p in `eval "$output_verbose_link_cmd"`; do case $prev$p in -L* | -R* | -l*) # Some compilers place space between "-{L,R}" and the path. # Remove the space. if test x-L = "$p" || test x-R = "$p"; then prev=$p continue fi # Expand the sysroot to ease extracting the directories later. if test -z "$prev"; then case $p in -L*) func_stripname_cnf '-L' '' "$p"; prev=-L; p=$func_stripname_result ;; -R*) func_stripname_cnf '-R' '' "$p"; prev=-R; p=$func_stripname_result ;; -l*) func_stripname_cnf '-l' '' "$p"; prev=-l; p=$func_stripname_result ;; esac fi case $p in =*) func_stripname_cnf '=' '' "$p"; p=$lt_sysroot$func_stripname_result ;; esac if test no = "$pre_test_object_deps_done"; then case $prev in -L | -R) # Internal compiler library paths should come after those # provided the user. The postdeps already come after the # user supplied libs so there is no need to process them. if test -z "$_LT_TAGVAR(compiler_lib_search_path, $1)"; then _LT_TAGVAR(compiler_lib_search_path, $1)=$prev$p else _LT_TAGVAR(compiler_lib_search_path, $1)="${_LT_TAGVAR(compiler_lib_search_path, $1)} $prev$p" fi ;; # The "-l" case would never come before the object being # linked, so don't bother handling this case. esac else if test -z "$_LT_TAGVAR(postdeps, $1)"; then _LT_TAGVAR(postdeps, $1)=$prev$p else _LT_TAGVAR(postdeps, $1)="${_LT_TAGVAR(postdeps, $1)} $prev$p" fi fi prev= ;; *.lto.$objext) ;; # Ignore GCC LTO objects *.$objext) # This assumes that the test object file only shows up # once in the compiler output. if test "$p" = "conftest.$objext"; then pre_test_object_deps_done=yes continue fi if test no = "$pre_test_object_deps_done"; then if test -z "$_LT_TAGVAR(predep_objects, $1)"; then _LT_TAGVAR(predep_objects, $1)=$p else _LT_TAGVAR(predep_objects, $1)="$_LT_TAGVAR(predep_objects, $1) $p" fi else if test -z "$_LT_TAGVAR(postdep_objects, $1)"; then _LT_TAGVAR(postdep_objects, $1)=$p else _LT_TAGVAR(postdep_objects, $1)="$_LT_TAGVAR(postdep_objects, $1) $p" fi fi ;; *) ;; # Ignore the rest. esac done # Clean up. rm -f a.out a.exe else echo "libtool.m4: error: problem compiling $1 test program" fi $RM -f confest.$objext CFLAGS=$_lt_libdeps_save_CFLAGS # PORTME: override above test on systems where it is broken m4_if([$1], [CXX], [case $host_os in interix[[3-9]]*) # Interix 3.5 installs completely hosed .la files for C++, so rather than # hack all around it, let's just trust "g++" to DTRT. _LT_TAGVAR(predep_objects,$1)= _LT_TAGVAR(postdep_objects,$1)= _LT_TAGVAR(postdeps,$1)= ;; esac ]) case " $_LT_TAGVAR(postdeps, $1) " in *" -lc "*) _LT_TAGVAR(archive_cmds_need_lc, $1)=no ;; esac _LT_TAGVAR(compiler_lib_search_dirs, $1)= if test -n "${_LT_TAGVAR(compiler_lib_search_path, $1)}"; then _LT_TAGVAR(compiler_lib_search_dirs, $1)=`echo " ${_LT_TAGVAR(compiler_lib_search_path, $1)}" | $SED -e 's! -L! !g' -e 's!^ !!'` fi _LT_TAGDECL([], [compiler_lib_search_dirs], [1], [The directories searched by this compiler when creating a shared library]) _LT_TAGDECL([], [predep_objects], [1], [Dependencies to place before and after the objects being linked to create a shared library]) _LT_TAGDECL([], [postdep_objects], [1]) _LT_TAGDECL([], [predeps], [1]) _LT_TAGDECL([], [postdeps], [1]) _LT_TAGDECL([], [compiler_lib_search_path], [1], [The library search path used internally by the compiler when linking a shared library]) ])# _LT_SYS_HIDDEN_LIBDEPS # _LT_LANG_F77_CONFIG([TAG]) # -------------------------- # Ensure that the configuration variables for a Fortran 77 compiler are # suitably defined. These variables are subsequently used by _LT_CONFIG # to write the compiler configuration to 'libtool'. m4_defun([_LT_LANG_F77_CONFIG], [AC_LANG_PUSH(Fortran 77) if test -z "$F77" || test no = "$F77"; then _lt_disable_F77=yes fi _LT_TAGVAR(archive_cmds_need_lc, $1)=no _LT_TAGVAR(allow_undefined_flag, $1)= _LT_TAGVAR(always_export_symbols, $1)=no _LT_TAGVAR(archive_expsym_cmds, $1)= _LT_TAGVAR(export_dynamic_flag_spec, $1)= _LT_TAGVAR(hardcode_direct, $1)=no _LT_TAGVAR(hardcode_direct_absolute, $1)=no _LT_TAGVAR(hardcode_libdir_flag_spec, $1)= _LT_TAGVAR(hardcode_libdir_separator, $1)= _LT_TAGVAR(hardcode_minus_L, $1)=no _LT_TAGVAR(hardcode_automatic, $1)=no _LT_TAGVAR(inherit_rpath, $1)=no _LT_TAGVAR(module_cmds, $1)= _LT_TAGVAR(module_expsym_cmds, $1)= _LT_TAGVAR(link_all_deplibs, $1)=unknown _LT_TAGVAR(old_archive_cmds, $1)=$old_archive_cmds _LT_TAGVAR(reload_flag, $1)=$reload_flag _LT_TAGVAR(reload_cmds, $1)=$reload_cmds _LT_TAGVAR(no_undefined_flag, $1)= _LT_TAGVAR(whole_archive_flag_spec, $1)= _LT_TAGVAR(enable_shared_with_static_runtimes, $1)=no # Source file extension for f77 test sources. ac_ext=f # Object file extension for compiled f77 test sources. objext=o _LT_TAGVAR(objext, $1)=$objext # No sense in running all these tests if we already determined that # the F77 compiler isn't working. Some variables (like enable_shared) # are currently assumed to apply to all compilers on this platform, # and will be corrupted by setting them based on a non-working compiler. if test yes != "$_lt_disable_F77"; then # Code to be used in simple compile tests lt_simple_compile_test_code="\ subroutine t return end " # Code to be used in simple link tests lt_simple_link_test_code="\ program t end " # ltmain only uses $CC for tagged configurations so make sure $CC is set. _LT_TAG_COMPILER # save warnings/boilerplate of simple test code _LT_COMPILER_BOILERPLATE _LT_LINKER_BOILERPLATE # Allow CC to be a program name with arguments. lt_save_CC=$CC lt_save_GCC=$GCC lt_save_CFLAGS=$CFLAGS CC=${F77-"f77"} CFLAGS=$FFLAGS compiler=$CC _LT_TAGVAR(compiler, $1)=$CC _LT_CC_BASENAME([$compiler]) GCC=$G77 if test -n "$compiler"; then AC_MSG_CHECKING([if libtool supports shared libraries]) AC_MSG_RESULT([$can_build_shared]) AC_MSG_CHECKING([whether to build shared libraries]) test no = "$can_build_shared" && enable_shared=no # On AIX, shared libraries and static libraries use the same namespace, and # are all built from PIC. case $host_os in aix3*) test yes = "$enable_shared" && enable_static=no if test -n "$RANLIB"; then archive_cmds="$archive_cmds~\$RANLIB \$lib" postinstall_cmds='$RANLIB $lib' fi ;; aix[[4-9]]*) if test ia64 != "$host_cpu"; then case $enable_shared,$with_aix_soname,$aix_use_runtimelinking in yes,aix,yes) ;; # shared object as lib.so file only yes,svr4,*) ;; # shared object as lib.so archive member only yes,*) enable_static=no ;; # shared object in lib.a archive as well esac fi ;; esac AC_MSG_RESULT([$enable_shared]) AC_MSG_CHECKING([whether to build static libraries]) # Make sure either enable_shared or enable_static is yes. test yes = "$enable_shared" || enable_static=yes AC_MSG_RESULT([$enable_static]) _LT_TAGVAR(GCC, $1)=$G77 _LT_TAGVAR(LD, $1)=$LD ## CAVEAT EMPTOR: ## There is no encapsulation within the following macros, do not change ## the running order or otherwise move them around unless you know exactly ## what you are doing... _LT_COMPILER_PIC($1) _LT_COMPILER_C_O($1) _LT_COMPILER_FILE_LOCKS($1) _LT_LINKER_SHLIBS($1) _LT_SYS_DYNAMIC_LINKER($1) _LT_LINKER_HARDCODE_LIBPATH($1) _LT_CONFIG($1) fi # test -n "$compiler" GCC=$lt_save_GCC CC=$lt_save_CC CFLAGS=$lt_save_CFLAGS fi # test yes != "$_lt_disable_F77" AC_LANG_POP ])# _LT_LANG_F77_CONFIG # _LT_LANG_FC_CONFIG([TAG]) # ------------------------- # Ensure that the configuration variables for a Fortran compiler are # suitably defined. These variables are subsequently used by _LT_CONFIG # to write the compiler configuration to 'libtool'. m4_defun([_LT_LANG_FC_CONFIG], [AC_LANG_PUSH(Fortran) if test -z "$FC" || test no = "$FC"; then _lt_disable_FC=yes fi _LT_TAGVAR(archive_cmds_need_lc, $1)=no _LT_TAGVAR(allow_undefined_flag, $1)= _LT_TAGVAR(always_export_symbols, $1)=no _LT_TAGVAR(archive_expsym_cmds, $1)= _LT_TAGVAR(export_dynamic_flag_spec, $1)= _LT_TAGVAR(hardcode_direct, $1)=no _LT_TAGVAR(hardcode_direct_absolute, $1)=no _LT_TAGVAR(hardcode_libdir_flag_spec, $1)= _LT_TAGVAR(hardcode_libdir_separator, $1)= _LT_TAGVAR(hardcode_minus_L, $1)=no _LT_TAGVAR(hardcode_automatic, $1)=no _LT_TAGVAR(inherit_rpath, $1)=no _LT_TAGVAR(module_cmds, $1)= _LT_TAGVAR(module_expsym_cmds, $1)= _LT_TAGVAR(link_all_deplibs, $1)=unknown _LT_TAGVAR(old_archive_cmds, $1)=$old_archive_cmds _LT_TAGVAR(reload_flag, $1)=$reload_flag _LT_TAGVAR(reload_cmds, $1)=$reload_cmds _LT_TAGVAR(no_undefined_flag, $1)= _LT_TAGVAR(whole_archive_flag_spec, $1)= _LT_TAGVAR(enable_shared_with_static_runtimes, $1)=no # Source file extension for fc test sources. ac_ext=${ac_fc_srcext-f} # Object file extension for compiled fc test sources. objext=o _LT_TAGVAR(objext, $1)=$objext # No sense in running all these tests if we already determined that # the FC compiler isn't working. Some variables (like enable_shared) # are currently assumed to apply to all compilers on this platform, # and will be corrupted by setting them based on a non-working compiler. if test yes != "$_lt_disable_FC"; then # Code to be used in simple compile tests lt_simple_compile_test_code="\ subroutine t return end " # Code to be used in simple link tests lt_simple_link_test_code="\ program t end " # ltmain only uses $CC for tagged configurations so make sure $CC is set. _LT_TAG_COMPILER # save warnings/boilerplate of simple test code _LT_COMPILER_BOILERPLATE _LT_LINKER_BOILERPLATE # Allow CC to be a program name with arguments. lt_save_CC=$CC lt_save_GCC=$GCC lt_save_CFLAGS=$CFLAGS CC=${FC-"f95"} CFLAGS=$FCFLAGS compiler=$CC GCC=$ac_cv_fc_compiler_gnu _LT_TAGVAR(compiler, $1)=$CC _LT_CC_BASENAME([$compiler]) if test -n "$compiler"; then AC_MSG_CHECKING([if libtool supports shared libraries]) AC_MSG_RESULT([$can_build_shared]) AC_MSG_CHECKING([whether to build shared libraries]) test no = "$can_build_shared" && enable_shared=no # On AIX, shared libraries and static libraries use the same namespace, and # are all built from PIC. case $host_os in aix3*) test yes = "$enable_shared" && enable_static=no if test -n "$RANLIB"; then archive_cmds="$archive_cmds~\$RANLIB \$lib" postinstall_cmds='$RANLIB $lib' fi ;; aix[[4-9]]*) if test ia64 != "$host_cpu"; then case $enable_shared,$with_aix_soname,$aix_use_runtimelinking in yes,aix,yes) ;; # shared object as lib.so file only yes,svr4,*) ;; # shared object as lib.so archive member only yes,*) enable_static=no ;; # shared object in lib.a archive as well esac fi ;; esac AC_MSG_RESULT([$enable_shared]) AC_MSG_CHECKING([whether to build static libraries]) # Make sure either enable_shared or enable_static is yes. test yes = "$enable_shared" || enable_static=yes AC_MSG_RESULT([$enable_static]) _LT_TAGVAR(GCC, $1)=$ac_cv_fc_compiler_gnu _LT_TAGVAR(LD, $1)=$LD ## CAVEAT EMPTOR: ## There is no encapsulation within the following macros, do not change ## the running order or otherwise move them around unless you know exactly ## what you are doing... _LT_SYS_HIDDEN_LIBDEPS($1) _LT_COMPILER_PIC($1) _LT_COMPILER_C_O($1) _LT_COMPILER_FILE_LOCKS($1) _LT_LINKER_SHLIBS($1) _LT_SYS_DYNAMIC_LINKER($1) _LT_LINKER_HARDCODE_LIBPATH($1) _LT_CONFIG($1) fi # test -n "$compiler" GCC=$lt_save_GCC CC=$lt_save_CC CFLAGS=$lt_save_CFLAGS fi # test yes != "$_lt_disable_FC" AC_LANG_POP ])# _LT_LANG_FC_CONFIG # _LT_LANG_GCJ_CONFIG([TAG]) # -------------------------- # Ensure that the configuration variables for the GNU Java Compiler compiler # are suitably defined. These variables are subsequently used by _LT_CONFIG # to write the compiler configuration to 'libtool'. m4_defun([_LT_LANG_GCJ_CONFIG], [AC_REQUIRE([LT_PROG_GCJ])dnl AC_LANG_SAVE # Source file extension for Java test sources. ac_ext=java # Object file extension for compiled Java test sources. objext=o _LT_TAGVAR(objext, $1)=$objext # Code to be used in simple compile tests lt_simple_compile_test_code="class foo {}" # Code to be used in simple link tests lt_simple_link_test_code='public class conftest { public static void main(String[[]] argv) {}; }' # ltmain only uses $CC for tagged configurations so make sure $CC is set. _LT_TAG_COMPILER # save warnings/boilerplate of simple test code _LT_COMPILER_BOILERPLATE _LT_LINKER_BOILERPLATE # Allow CC to be a program name with arguments. lt_save_CC=$CC lt_save_CFLAGS=$CFLAGS lt_save_GCC=$GCC GCC=yes CC=${GCJ-"gcj"} CFLAGS=$GCJFLAGS compiler=$CC _LT_TAGVAR(compiler, $1)=$CC _LT_TAGVAR(LD, $1)=$LD _LT_CC_BASENAME([$compiler]) # GCJ did not exist at the time GCC didn't implicitly link libc in. _LT_TAGVAR(archive_cmds_need_lc, $1)=no _LT_TAGVAR(old_archive_cmds, $1)=$old_archive_cmds _LT_TAGVAR(reload_flag, $1)=$reload_flag _LT_TAGVAR(reload_cmds, $1)=$reload_cmds ## CAVEAT EMPTOR: ## There is no encapsulation within the following macros, do not change ## the running order or otherwise move them around unless you know exactly ## what you are doing... if test -n "$compiler"; then _LT_COMPILER_NO_RTTI($1) _LT_COMPILER_PIC($1) _LT_COMPILER_C_O($1) _LT_COMPILER_FILE_LOCKS($1) _LT_LINKER_SHLIBS($1) _LT_LINKER_HARDCODE_LIBPATH($1) _LT_CONFIG($1) fi AC_LANG_RESTORE GCC=$lt_save_GCC CC=$lt_save_CC CFLAGS=$lt_save_CFLAGS ])# _LT_LANG_GCJ_CONFIG # _LT_LANG_GO_CONFIG([TAG]) # -------------------------- # Ensure that the configuration variables for the GNU Go compiler # are suitably defined. These variables are subsequently used by _LT_CONFIG # to write the compiler configuration to 'libtool'. m4_defun([_LT_LANG_GO_CONFIG], [AC_REQUIRE([LT_PROG_GO])dnl AC_LANG_SAVE # Source file extension for Go test sources. ac_ext=go # Object file extension for compiled Go test sources. objext=o _LT_TAGVAR(objext, $1)=$objext # Code to be used in simple compile tests lt_simple_compile_test_code="package main; func main() { }" # Code to be used in simple link tests lt_simple_link_test_code='package main; func main() { }' # ltmain only uses $CC for tagged configurations so make sure $CC is set. _LT_TAG_COMPILER # save warnings/boilerplate of simple test code _LT_COMPILER_BOILERPLATE _LT_LINKER_BOILERPLATE # Allow CC to be a program name with arguments. lt_save_CC=$CC lt_save_CFLAGS=$CFLAGS lt_save_GCC=$GCC GCC=yes CC=${GOC-"gccgo"} CFLAGS=$GOFLAGS compiler=$CC _LT_TAGVAR(compiler, $1)=$CC _LT_TAGVAR(LD, $1)=$LD _LT_CC_BASENAME([$compiler]) # Go did not exist at the time GCC didn't implicitly link libc in. _LT_TAGVAR(archive_cmds_need_lc, $1)=no _LT_TAGVAR(old_archive_cmds, $1)=$old_archive_cmds _LT_TAGVAR(reload_flag, $1)=$reload_flag _LT_TAGVAR(reload_cmds, $1)=$reload_cmds ## CAVEAT EMPTOR: ## There is no encapsulation within the following macros, do not change ## the running order or otherwise move them around unless you know exactly ## what you are doing... if test -n "$compiler"; then _LT_COMPILER_NO_RTTI($1) _LT_COMPILER_PIC($1) _LT_COMPILER_C_O($1) _LT_COMPILER_FILE_LOCKS($1) _LT_LINKER_SHLIBS($1) _LT_LINKER_HARDCODE_LIBPATH($1) _LT_CONFIG($1) fi AC_LANG_RESTORE GCC=$lt_save_GCC CC=$lt_save_CC CFLAGS=$lt_save_CFLAGS ])# _LT_LANG_GO_CONFIG # _LT_LANG_RC_CONFIG([TAG]) # ------------------------- # Ensure that the configuration variables for the Windows resource compiler # are suitably defined. These variables are subsequently used by _LT_CONFIG # to write the compiler configuration to 'libtool'. m4_defun([_LT_LANG_RC_CONFIG], [AC_REQUIRE([LT_PROG_RC])dnl AC_LANG_SAVE # Source file extension for RC test sources. ac_ext=rc # Object file extension for compiled RC test sources. objext=o _LT_TAGVAR(objext, $1)=$objext # Code to be used in simple compile tests lt_simple_compile_test_code='sample MENU { MENUITEM "&Soup", 100, CHECKED }' # Code to be used in simple link tests lt_simple_link_test_code=$lt_simple_compile_test_code # ltmain only uses $CC for tagged configurations so make sure $CC is set. _LT_TAG_COMPILER # save warnings/boilerplate of simple test code _LT_COMPILER_BOILERPLATE _LT_LINKER_BOILERPLATE # Allow CC to be a program name with arguments. lt_save_CC=$CC lt_save_CFLAGS=$CFLAGS lt_save_GCC=$GCC GCC= CC=${RC-"windres"} CFLAGS= compiler=$CC _LT_TAGVAR(compiler, $1)=$CC _LT_CC_BASENAME([$compiler]) _LT_TAGVAR(lt_cv_prog_compiler_c_o, $1)=yes if test -n "$compiler"; then : _LT_CONFIG($1) fi GCC=$lt_save_GCC AC_LANG_RESTORE CC=$lt_save_CC CFLAGS=$lt_save_CFLAGS ])# _LT_LANG_RC_CONFIG # LT_PROG_GCJ # ----------- AC_DEFUN([LT_PROG_GCJ], [m4_ifdef([AC_PROG_GCJ], [AC_PROG_GCJ], [m4_ifdef([A][M_PROG_GCJ], [A][M_PROG_GCJ], [AC_CHECK_TOOL(GCJ, gcj,) test set = "${GCJFLAGS+set}" || GCJFLAGS="-g -O2" AC_SUBST(GCJFLAGS)])])[]dnl ]) # Old name: AU_ALIAS([LT_AC_PROG_GCJ], [LT_PROG_GCJ]) dnl aclocal-1.4 backwards compatibility: dnl AC_DEFUN([LT_AC_PROG_GCJ], []) # LT_PROG_GO # ---------- AC_DEFUN([LT_PROG_GO], [AC_CHECK_TOOL(GOC, gccgo,) ]) # LT_PROG_RC # ---------- AC_DEFUN([LT_PROG_RC], [AC_CHECK_TOOL(RC, windres,) ]) # Old name: AU_ALIAS([LT_AC_PROG_RC], [LT_PROG_RC]) dnl aclocal-1.4 backwards compatibility: dnl AC_DEFUN([LT_AC_PROG_RC], []) # _LT_DECL_EGREP # -------------- # If we don't have a new enough Autoconf to choose the best grep # available, choose the one first in the user's PATH. m4_defun([_LT_DECL_EGREP], [AC_REQUIRE([AC_PROG_EGREP])dnl AC_REQUIRE([AC_PROG_FGREP])dnl test -z "$GREP" && GREP=grep _LT_DECL([], [GREP], [1], [A grep program that handles long lines]) _LT_DECL([], [EGREP], [1], [An ERE matcher]) _LT_DECL([], [FGREP], [1], [A literal string matcher]) dnl Non-bleeding-edge autoconf doesn't subst GREP, so do it here too AC_SUBST([GREP]) ]) # _LT_DECL_OBJDUMP # -------------- # If we don't have a new enough Autoconf to choose the best objdump # available, choose the one first in the user's PATH. m4_defun([_LT_DECL_OBJDUMP], [AC_CHECK_TOOL(OBJDUMP, objdump, false) test -z "$OBJDUMP" && OBJDUMP=objdump _LT_DECL([], [OBJDUMP], [1], [An object symbol dumper]) AC_SUBST([OBJDUMP]) ]) # _LT_DECL_DLLTOOL # ---------------- # Ensure DLLTOOL variable is set. m4_defun([_LT_DECL_DLLTOOL], [AC_CHECK_TOOL(DLLTOOL, dlltool, false) test -z "$DLLTOOL" && DLLTOOL=dlltool _LT_DECL([], [DLLTOOL], [1], [DLL creation program]) AC_SUBST([DLLTOOL]) ]) # _LT_DECL_SED # ------------ # Check for a fully-functional sed program, that truncates # as few characters as possible. Prefer GNU sed if found. m4_defun([_LT_DECL_SED], [AC_PROG_SED test -z "$SED" && SED=sed Xsed="$SED -e 1s/^X//" _LT_DECL([], [SED], [1], [A sed program that does not truncate output]) _LT_DECL([], [Xsed], ["\$SED -e 1s/^X//"], [Sed that helps us avoid accidentally triggering echo(1) options like -n]) ])# _LT_DECL_SED m4_ifndef([AC_PROG_SED], [ ############################################################ # NOTE: This macro has been submitted for inclusion into # # GNU Autoconf as AC_PROG_SED. When it is available in # # a released version of Autoconf we should remove this # # macro and use it instead. # ############################################################ m4_defun([AC_PROG_SED], [AC_MSG_CHECKING([for a sed that does not truncate output]) AC_CACHE_VAL(lt_cv_path_SED, [# Loop through the user's path and test for sed and gsed. # Then use that list of sed's as ones to test for truncation. as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for lt_ac_prog in sed gsed; do for ac_exec_ext in '' $ac_executable_extensions; do if $as_executable_p "$as_dir/$lt_ac_prog$ac_exec_ext"; then lt_ac_sed_list="$lt_ac_sed_list $as_dir/$lt_ac_prog$ac_exec_ext" fi done done done IFS=$as_save_IFS lt_ac_max=0 lt_ac_count=0 # Add /usr/xpg4/bin/sed as it is typically found on Solaris # along with /bin/sed that truncates output. for lt_ac_sed in $lt_ac_sed_list /usr/xpg4/bin/sed; do test ! -f "$lt_ac_sed" && continue cat /dev/null > conftest.in lt_ac_count=0 echo $ECHO_N "0123456789$ECHO_C" >conftest.in # Check for GNU sed and select it if it is found. if "$lt_ac_sed" --version 2>&1 < /dev/null | grep 'GNU' > /dev/null; then lt_cv_path_SED=$lt_ac_sed break fi while true; do cat conftest.in conftest.in >conftest.tmp mv conftest.tmp conftest.in cp conftest.in conftest.nl echo >>conftest.nl $lt_ac_sed -e 's/a$//' < conftest.nl >conftest.out || break cmp -s conftest.out conftest.nl || break # 10000 chars as input seems more than enough test 10 -lt "$lt_ac_count" && break lt_ac_count=`expr $lt_ac_count + 1` if test "$lt_ac_count" -gt "$lt_ac_max"; then lt_ac_max=$lt_ac_count lt_cv_path_SED=$lt_ac_sed fi done done ]) SED=$lt_cv_path_SED AC_SUBST([SED]) AC_MSG_RESULT([$SED]) ])#AC_PROG_SED ])#m4_ifndef # Old name: AU_ALIAS([LT_AC_PROG_SED], [AC_PROG_SED]) dnl aclocal-1.4 backwards compatibility: dnl AC_DEFUN([LT_AC_PROG_SED], []) # _LT_CHECK_SHELL_FEATURES # ------------------------ # Find out whether the shell is Bourne or XSI compatible, # or has some other useful features. m4_defun([_LT_CHECK_SHELL_FEATURES], [if ( (MAIL=60; unset MAIL) || exit) >/dev/null 2>&1; then lt_unset=unset else lt_unset=false fi _LT_DECL([], [lt_unset], [0], [whether the shell understands "unset"])dnl # test EBCDIC or ASCII case `echo X|tr X '\101'` in A) # ASCII based system # \n is not interpreted correctly by Solaris 8 /usr/ucb/tr lt_SP2NL='tr \040 \012' lt_NL2SP='tr \015\012 \040\040' ;; *) # EBCDIC based system lt_SP2NL='tr \100 \n' lt_NL2SP='tr \r\n \100\100' ;; esac _LT_DECL([SP2NL], [lt_SP2NL], [1], [turn spaces into newlines])dnl _LT_DECL([NL2SP], [lt_NL2SP], [1], [turn newlines into spaces])dnl ])# _LT_CHECK_SHELL_FEATURES # _LT_PATH_CONVERSION_FUNCTIONS # ----------------------------- # Determine what file name conversion functions should be used by # func_to_host_file (and, implicitly, by func_to_host_path). These are needed # for certain cross-compile configurations and native mingw. m4_defun([_LT_PATH_CONVERSION_FUNCTIONS], [AC_REQUIRE([AC_CANONICAL_HOST])dnl AC_REQUIRE([AC_CANONICAL_BUILD])dnl AC_MSG_CHECKING([how to convert $build file names to $host format]) AC_CACHE_VAL(lt_cv_to_host_file_cmd, [case $host in *-*-mingw* ) case $build in *-*-mingw* ) # actually msys lt_cv_to_host_file_cmd=func_convert_file_msys_to_w32 ;; *-*-cygwin* ) lt_cv_to_host_file_cmd=func_convert_file_cygwin_to_w32 ;; * ) # otherwise, assume *nix lt_cv_to_host_file_cmd=func_convert_file_nix_to_w32 ;; esac ;; *-*-cygwin* ) case $build in *-*-mingw* ) # actually msys lt_cv_to_host_file_cmd=func_convert_file_msys_to_cygwin ;; *-*-cygwin* ) lt_cv_to_host_file_cmd=func_convert_file_noop ;; * ) # otherwise, assume *nix lt_cv_to_host_file_cmd=func_convert_file_nix_to_cygwin ;; esac ;; * ) # unhandled hosts (and "normal" native builds) lt_cv_to_host_file_cmd=func_convert_file_noop ;; esac ]) to_host_file_cmd=$lt_cv_to_host_file_cmd AC_MSG_RESULT([$lt_cv_to_host_file_cmd]) _LT_DECL([to_host_file_cmd], [lt_cv_to_host_file_cmd], [0], [convert $build file names to $host format])dnl AC_MSG_CHECKING([how to convert $build file names to toolchain format]) AC_CACHE_VAL(lt_cv_to_tool_file_cmd, [#assume ordinary cross tools, or native build. lt_cv_to_tool_file_cmd=func_convert_file_noop case $host in *-*-mingw* ) case $build in *-*-mingw* ) # actually msys lt_cv_to_tool_file_cmd=func_convert_file_msys_to_w32 ;; esac ;; esac ]) to_tool_file_cmd=$lt_cv_to_tool_file_cmd AC_MSG_RESULT([$lt_cv_to_tool_file_cmd]) _LT_DECL([to_tool_file_cmd], [lt_cv_to_tool_file_cmd], [0], [convert $build files to toolchain format])dnl ])# _LT_PATH_CONVERSION_FUNCTIONS vowpal-wabbit-8.5.0.dfsg1/acinclude.d/ltoptions.m4000066400000000000000000000342621323713302100217550ustar00rootroot00000000000000# Helper functions for option handling. -*- Autoconf -*- # # Copyright (C) 2004-2005, 2007-2009, 2011-2015 Free Software # Foundation, Inc. # Written by Gary V. Vaughan, 2004 # # This file is free software; the Free Software Foundation gives # unlimited permission to copy and/or distribute it, with or without # modifications, as long as this notice is preserved. # serial 8 ltoptions.m4 # This is to help aclocal find these macros, as it can't see m4_define. AC_DEFUN([LTOPTIONS_VERSION], [m4_if([1])]) # _LT_MANGLE_OPTION(MACRO-NAME, OPTION-NAME) # ------------------------------------------ m4_define([_LT_MANGLE_OPTION], [[_LT_OPTION_]m4_bpatsubst($1__$2, [[^a-zA-Z0-9_]], [_])]) # _LT_SET_OPTION(MACRO-NAME, OPTION-NAME) # --------------------------------------- # Set option OPTION-NAME for macro MACRO-NAME, and if there is a # matching handler defined, dispatch to it. Other OPTION-NAMEs are # saved as a flag. m4_define([_LT_SET_OPTION], [m4_define(_LT_MANGLE_OPTION([$1], [$2]))dnl m4_ifdef(_LT_MANGLE_DEFUN([$1], [$2]), _LT_MANGLE_DEFUN([$1], [$2]), [m4_warning([Unknown $1 option '$2'])])[]dnl ]) # _LT_IF_OPTION(MACRO-NAME, OPTION-NAME, IF-SET, [IF-NOT-SET]) # ------------------------------------------------------------ # Execute IF-SET if OPTION is set, IF-NOT-SET otherwise. m4_define([_LT_IF_OPTION], [m4_ifdef(_LT_MANGLE_OPTION([$1], [$2]), [$3], [$4])]) # _LT_UNLESS_OPTIONS(MACRO-NAME, OPTION-LIST, IF-NOT-SET) # ------------------------------------------------------- # Execute IF-NOT-SET unless all options in OPTION-LIST for MACRO-NAME # are set. m4_define([_LT_UNLESS_OPTIONS], [m4_foreach([_LT_Option], m4_split(m4_normalize([$2])), [m4_ifdef(_LT_MANGLE_OPTION([$1], _LT_Option), [m4_define([$0_found])])])[]dnl m4_ifdef([$0_found], [m4_undefine([$0_found])], [$3 ])[]dnl ]) # _LT_SET_OPTIONS(MACRO-NAME, OPTION-LIST) # ---------------------------------------- # OPTION-LIST is a space-separated list of Libtool options associated # with MACRO-NAME. If any OPTION has a matching handler declared with # LT_OPTION_DEFINE, dispatch to that macro; otherwise complain about # the unknown option and exit. m4_defun([_LT_SET_OPTIONS], [# Set options m4_foreach([_LT_Option], m4_split(m4_normalize([$2])), [_LT_SET_OPTION([$1], _LT_Option)]) m4_if([$1],[LT_INIT],[ dnl dnl Simply set some default values (i.e off) if boolean options were not dnl specified: _LT_UNLESS_OPTIONS([LT_INIT], [dlopen], [enable_dlopen=no ]) _LT_UNLESS_OPTIONS([LT_INIT], [win32-dll], [enable_win32_dll=no ]) dnl dnl If no reference was made to various pairs of opposing options, then dnl we run the default mode handler for the pair. For example, if neither dnl 'shared' nor 'disable-shared' was passed, we enable building of shared dnl archives by default: _LT_UNLESS_OPTIONS([LT_INIT], [shared disable-shared], [_LT_ENABLE_SHARED]) _LT_UNLESS_OPTIONS([LT_INIT], [static disable-static], [_LT_ENABLE_STATIC]) _LT_UNLESS_OPTIONS([LT_INIT], [pic-only no-pic], [_LT_WITH_PIC]) _LT_UNLESS_OPTIONS([LT_INIT], [fast-install disable-fast-install], [_LT_ENABLE_FAST_INSTALL]) _LT_UNLESS_OPTIONS([LT_INIT], [aix-soname=aix aix-soname=both aix-soname=svr4], [_LT_WITH_AIX_SONAME([aix])]) ]) ])# _LT_SET_OPTIONS ## --------------------------------- ## ## Macros to handle LT_INIT options. ## ## --------------------------------- ## # _LT_MANGLE_DEFUN(MACRO-NAME, OPTION-NAME) # ----------------------------------------- m4_define([_LT_MANGLE_DEFUN], [[_LT_OPTION_DEFUN_]m4_bpatsubst(m4_toupper([$1__$2]), [[^A-Z0-9_]], [_])]) # LT_OPTION_DEFINE(MACRO-NAME, OPTION-NAME, CODE) # ----------------------------------------------- m4_define([LT_OPTION_DEFINE], [m4_define(_LT_MANGLE_DEFUN([$1], [$2]), [$3])[]dnl ])# LT_OPTION_DEFINE # dlopen # ------ LT_OPTION_DEFINE([LT_INIT], [dlopen], [enable_dlopen=yes ]) AU_DEFUN([AC_LIBTOOL_DLOPEN], [_LT_SET_OPTION([LT_INIT], [dlopen]) AC_DIAGNOSE([obsolete], [$0: Remove this warning and the call to _LT_SET_OPTION when you put the 'dlopen' option into LT_INIT's first parameter.]) ]) dnl aclocal-1.4 backwards compatibility: dnl AC_DEFUN([AC_LIBTOOL_DLOPEN], []) # win32-dll # --------- # Declare package support for building win32 dll's. LT_OPTION_DEFINE([LT_INIT], [win32-dll], [enable_win32_dll=yes case $host in *-*-cygwin* | *-*-mingw* | *-*-pw32* | *-*-cegcc*) AC_CHECK_TOOL(AS, as, false) AC_CHECK_TOOL(DLLTOOL, dlltool, false) AC_CHECK_TOOL(OBJDUMP, objdump, false) ;; esac test -z "$AS" && AS=as _LT_DECL([], [AS], [1], [Assembler program])dnl test -z "$DLLTOOL" && DLLTOOL=dlltool _LT_DECL([], [DLLTOOL], [1], [DLL creation program])dnl test -z "$OBJDUMP" && OBJDUMP=objdump _LT_DECL([], [OBJDUMP], [1], [Object dumper program])dnl ])# win32-dll AU_DEFUN([AC_LIBTOOL_WIN32_DLL], [AC_REQUIRE([AC_CANONICAL_HOST])dnl _LT_SET_OPTION([LT_INIT], [win32-dll]) AC_DIAGNOSE([obsolete], [$0: Remove this warning and the call to _LT_SET_OPTION when you put the 'win32-dll' option into LT_INIT's first parameter.]) ]) dnl aclocal-1.4 backwards compatibility: dnl AC_DEFUN([AC_LIBTOOL_WIN32_DLL], []) # _LT_ENABLE_SHARED([DEFAULT]) # ---------------------------- # implement the --enable-shared flag, and supports the 'shared' and # 'disable-shared' LT_INIT options. # DEFAULT is either 'yes' or 'no'. If omitted, it defaults to 'yes'. m4_define([_LT_ENABLE_SHARED], [m4_define([_LT_ENABLE_SHARED_DEFAULT], [m4_if($1, no, no, yes)])dnl AC_ARG_ENABLE([shared], [AS_HELP_STRING([--enable-shared@<:@=PKGS@:>@], [build shared libraries @<:@default=]_LT_ENABLE_SHARED_DEFAULT[@:>@])], [p=${PACKAGE-default} case $enableval in yes) enable_shared=yes ;; no) enable_shared=no ;; *) enable_shared=no # Look at the argument we got. We use all the common list separators. lt_save_ifs=$IFS; IFS=$IFS$PATH_SEPARATOR, for pkg in $enableval; do IFS=$lt_save_ifs if test "X$pkg" = "X$p"; then enable_shared=yes fi done IFS=$lt_save_ifs ;; esac], [enable_shared=]_LT_ENABLE_SHARED_DEFAULT) _LT_DECL([build_libtool_libs], [enable_shared], [0], [Whether or not to build shared libraries]) ])# _LT_ENABLE_SHARED LT_OPTION_DEFINE([LT_INIT], [shared], [_LT_ENABLE_SHARED([yes])]) LT_OPTION_DEFINE([LT_INIT], [disable-shared], [_LT_ENABLE_SHARED([no])]) # Old names: AC_DEFUN([AC_ENABLE_SHARED], [_LT_SET_OPTION([LT_INIT], m4_if([$1], [no], [disable-])[shared]) ]) AC_DEFUN([AC_DISABLE_SHARED], [_LT_SET_OPTION([LT_INIT], [disable-shared]) ]) AU_DEFUN([AM_ENABLE_SHARED], [AC_ENABLE_SHARED($@)]) AU_DEFUN([AM_DISABLE_SHARED], [AC_DISABLE_SHARED($@)]) dnl aclocal-1.4 backwards compatibility: dnl AC_DEFUN([AM_ENABLE_SHARED], []) dnl AC_DEFUN([AM_DISABLE_SHARED], []) # _LT_ENABLE_STATIC([DEFAULT]) # ---------------------------- # implement the --enable-static flag, and support the 'static' and # 'disable-static' LT_INIT options. # DEFAULT is either 'yes' or 'no'. If omitted, it defaults to 'yes'. m4_define([_LT_ENABLE_STATIC], [m4_define([_LT_ENABLE_STATIC_DEFAULT], [m4_if($1, no, no, yes)])dnl AC_ARG_ENABLE([static], [AS_HELP_STRING([--enable-static@<:@=PKGS@:>@], [build static libraries @<:@default=]_LT_ENABLE_STATIC_DEFAULT[@:>@])], [p=${PACKAGE-default} case $enableval in yes) enable_static=yes ;; no) enable_static=no ;; *) enable_static=no # Look at the argument we got. We use all the common list separators. lt_save_ifs=$IFS; IFS=$IFS$PATH_SEPARATOR, for pkg in $enableval; do IFS=$lt_save_ifs if test "X$pkg" = "X$p"; then enable_static=yes fi done IFS=$lt_save_ifs ;; esac], [enable_static=]_LT_ENABLE_STATIC_DEFAULT) _LT_DECL([build_old_libs], [enable_static], [0], [Whether or not to build static libraries]) ])# _LT_ENABLE_STATIC LT_OPTION_DEFINE([LT_INIT], [static], [_LT_ENABLE_STATIC([yes])]) LT_OPTION_DEFINE([LT_INIT], [disable-static], [_LT_ENABLE_STATIC([no])]) # Old names: AC_DEFUN([AC_ENABLE_STATIC], [_LT_SET_OPTION([LT_INIT], m4_if([$1], [no], [disable-])[static]) ]) AC_DEFUN([AC_DISABLE_STATIC], [_LT_SET_OPTION([LT_INIT], [disable-static]) ]) AU_DEFUN([AM_ENABLE_STATIC], [AC_ENABLE_STATIC($@)]) AU_DEFUN([AM_DISABLE_STATIC], [AC_DISABLE_STATIC($@)]) dnl aclocal-1.4 backwards compatibility: dnl AC_DEFUN([AM_ENABLE_STATIC], []) dnl AC_DEFUN([AM_DISABLE_STATIC], []) # _LT_ENABLE_FAST_INSTALL([DEFAULT]) # ---------------------------------- # implement the --enable-fast-install flag, and support the 'fast-install' # and 'disable-fast-install' LT_INIT options. # DEFAULT is either 'yes' or 'no'. If omitted, it defaults to 'yes'. m4_define([_LT_ENABLE_FAST_INSTALL], [m4_define([_LT_ENABLE_FAST_INSTALL_DEFAULT], [m4_if($1, no, no, yes)])dnl AC_ARG_ENABLE([fast-install], [AS_HELP_STRING([--enable-fast-install@<:@=PKGS@:>@], [optimize for fast installation @<:@default=]_LT_ENABLE_FAST_INSTALL_DEFAULT[@:>@])], [p=${PACKAGE-default} case $enableval in yes) enable_fast_install=yes ;; no) enable_fast_install=no ;; *) enable_fast_install=no # Look at the argument we got. We use all the common list separators. lt_save_ifs=$IFS; IFS=$IFS$PATH_SEPARATOR, for pkg in $enableval; do IFS=$lt_save_ifs if test "X$pkg" = "X$p"; then enable_fast_install=yes fi done IFS=$lt_save_ifs ;; esac], [enable_fast_install=]_LT_ENABLE_FAST_INSTALL_DEFAULT) _LT_DECL([fast_install], [enable_fast_install], [0], [Whether or not to optimize for fast installation])dnl ])# _LT_ENABLE_FAST_INSTALL LT_OPTION_DEFINE([LT_INIT], [fast-install], [_LT_ENABLE_FAST_INSTALL([yes])]) LT_OPTION_DEFINE([LT_INIT], [disable-fast-install], [_LT_ENABLE_FAST_INSTALL([no])]) # Old names: AU_DEFUN([AC_ENABLE_FAST_INSTALL], [_LT_SET_OPTION([LT_INIT], m4_if([$1], [no], [disable-])[fast-install]) AC_DIAGNOSE([obsolete], [$0: Remove this warning and the call to _LT_SET_OPTION when you put the 'fast-install' option into LT_INIT's first parameter.]) ]) AU_DEFUN([AC_DISABLE_FAST_INSTALL], [_LT_SET_OPTION([LT_INIT], [disable-fast-install]) AC_DIAGNOSE([obsolete], [$0: Remove this warning and the call to _LT_SET_OPTION when you put the 'disable-fast-install' option into LT_INIT's first parameter.]) ]) dnl aclocal-1.4 backwards compatibility: dnl AC_DEFUN([AC_ENABLE_FAST_INSTALL], []) dnl AC_DEFUN([AM_DISABLE_FAST_INSTALL], []) # _LT_WITH_AIX_SONAME([DEFAULT]) # ---------------------------------- # implement the --with-aix-soname flag, and support the `aix-soname=aix' # and `aix-soname=both' and `aix-soname=svr4' LT_INIT options. DEFAULT # is either `aix', `both' or `svr4'. If omitted, it defaults to `aix'. m4_define([_LT_WITH_AIX_SONAME], [m4_define([_LT_WITH_AIX_SONAME_DEFAULT], [m4_if($1, svr4, svr4, m4_if($1, both, both, aix))])dnl shared_archive_member_spec= case $host,$enable_shared in power*-*-aix[[5-9]]*,yes) AC_MSG_CHECKING([which variant of shared library versioning to provide]) AC_ARG_WITH([aix-soname], [AS_HELP_STRING([--with-aix-soname=aix|svr4|both], [shared library versioning (aka "SONAME") variant to provide on AIX, @<:@default=]_LT_WITH_AIX_SONAME_DEFAULT[@:>@.])], [case $withval in aix|svr4|both) ;; *) AC_MSG_ERROR([Unknown argument to --with-aix-soname]) ;; esac lt_cv_with_aix_soname=$with_aix_soname], [AC_CACHE_VAL([lt_cv_with_aix_soname], [lt_cv_with_aix_soname=]_LT_WITH_AIX_SONAME_DEFAULT) with_aix_soname=$lt_cv_with_aix_soname]) AC_MSG_RESULT([$with_aix_soname]) if test aix != "$with_aix_soname"; then # For the AIX way of multilib, we name the shared archive member # based on the bitwidth used, traditionally 'shr.o' or 'shr_64.o', # and 'shr.imp' or 'shr_64.imp', respectively, for the Import File. # Even when GNU compilers ignore OBJECT_MODE but need '-maix64' flag, # the AIX toolchain works better with OBJECT_MODE set (default 32). if test 64 = "${OBJECT_MODE-32}"; then shared_archive_member_spec=shr_64 else shared_archive_member_spec=shr fi fi ;; *) with_aix_soname=aix ;; esac _LT_DECL([], [shared_archive_member_spec], [0], [Shared archive member basename, for filename based shared library versioning on AIX])dnl ])# _LT_WITH_AIX_SONAME LT_OPTION_DEFINE([LT_INIT], [aix-soname=aix], [_LT_WITH_AIX_SONAME([aix])]) LT_OPTION_DEFINE([LT_INIT], [aix-soname=both], [_LT_WITH_AIX_SONAME([both])]) LT_OPTION_DEFINE([LT_INIT], [aix-soname=svr4], [_LT_WITH_AIX_SONAME([svr4])]) # _LT_WITH_PIC([MODE]) # -------------------- # implement the --with-pic flag, and support the 'pic-only' and 'no-pic' # LT_INIT options. # MODE is either 'yes' or 'no'. If omitted, it defaults to 'both'. m4_define([_LT_WITH_PIC], [AC_ARG_WITH([pic], [AS_HELP_STRING([--with-pic@<:@=PKGS@:>@], [try to use only PIC/non-PIC objects @<:@default=use both@:>@])], [lt_p=${PACKAGE-default} case $withval in yes|no) pic_mode=$withval ;; *) pic_mode=default # Look at the argument we got. We use all the common list separators. lt_save_ifs=$IFS; IFS=$IFS$PATH_SEPARATOR, for lt_pkg in $withval; do IFS=$lt_save_ifs if test "X$lt_pkg" = "X$lt_p"; then pic_mode=yes fi done IFS=$lt_save_ifs ;; esac], [pic_mode=m4_default([$1], [default])]) _LT_DECL([], [pic_mode], [0], [What type of objects to build])dnl ])# _LT_WITH_PIC LT_OPTION_DEFINE([LT_INIT], [pic-only], [_LT_WITH_PIC([yes])]) LT_OPTION_DEFINE([LT_INIT], [no-pic], [_LT_WITH_PIC([no])]) # Old name: AU_DEFUN([AC_LIBTOOL_PICMODE], [_LT_SET_OPTION([LT_INIT], [pic-only]) AC_DIAGNOSE([obsolete], [$0: Remove this warning and the call to _LT_SET_OPTION when you put the 'pic-only' option into LT_INIT's first parameter.]) ]) dnl aclocal-1.4 backwards compatibility: dnl AC_DEFUN([AC_LIBTOOL_PICMODE], []) ## ----------------- ## ## LTDL_INIT Options ## ## ----------------- ## m4_define([_LTDL_MODE], []) LT_OPTION_DEFINE([LTDL_INIT], [nonrecursive], [m4_define([_LTDL_MODE], [nonrecursive])]) LT_OPTION_DEFINE([LTDL_INIT], [recursive], [m4_define([_LTDL_MODE], [recursive])]) LT_OPTION_DEFINE([LTDL_INIT], [subproject], [m4_define([_LTDL_MODE], [subproject])]) m4_define([_LTDL_TYPE], []) LT_OPTION_DEFINE([LTDL_INIT], [installable], [m4_define([_LTDL_TYPE], [installable])]) LT_OPTION_DEFINE([LTDL_INIT], [convenience], [m4_define([_LTDL_TYPE], [convenience])]) vowpal-wabbit-8.5.0.dfsg1/acinclude.d/ltsugar.m4000066400000000000000000000104401323713302100213730ustar00rootroot00000000000000# ltsugar.m4 -- libtool m4 base layer. -*-Autoconf-*- # # Copyright (C) 2004-2005, 2007-2008, 2011-2015 Free Software # Foundation, Inc. # Written by Gary V. Vaughan, 2004 # # This file is free software; the Free Software Foundation gives # unlimited permission to copy and/or distribute it, with or without # modifications, as long as this notice is preserved. # serial 6 ltsugar.m4 # This is to help aclocal find these macros, as it can't see m4_define. AC_DEFUN([LTSUGAR_VERSION], [m4_if([0.1])]) # lt_join(SEP, ARG1, [ARG2...]) # ----------------------------- # Produce ARG1SEPARG2...SEPARGn, omitting [] arguments and their # associated separator. # Needed until we can rely on m4_join from Autoconf 2.62, since all earlier # versions in m4sugar had bugs. m4_define([lt_join], [m4_if([$#], [1], [], [$#], [2], [[$2]], [m4_if([$2], [], [], [[$2]_])$0([$1], m4_shift(m4_shift($@)))])]) m4_define([_lt_join], [m4_if([$#$2], [2], [], [m4_if([$2], [], [], [[$1$2]])$0([$1], m4_shift(m4_shift($@)))])]) # lt_car(LIST) # lt_cdr(LIST) # ------------ # Manipulate m4 lists. # These macros are necessary as long as will still need to support # Autoconf-2.59, which quotes differently. m4_define([lt_car], [[$1]]) m4_define([lt_cdr], [m4_if([$#], 0, [m4_fatal([$0: cannot be called without arguments])], [$#], 1, [], [m4_dquote(m4_shift($@))])]) m4_define([lt_unquote], $1) # lt_append(MACRO-NAME, STRING, [SEPARATOR]) # ------------------------------------------ # Redefine MACRO-NAME to hold its former content plus 'SEPARATOR''STRING'. # Note that neither SEPARATOR nor STRING are expanded; they are appended # to MACRO-NAME as is (leaving the expansion for when MACRO-NAME is invoked). # No SEPARATOR is output if MACRO-NAME was previously undefined (different # than defined and empty). # # This macro is needed until we can rely on Autoconf 2.62, since earlier # versions of m4sugar mistakenly expanded SEPARATOR but not STRING. m4_define([lt_append], [m4_define([$1], m4_ifdef([$1], [m4_defn([$1])[$3]])[$2])]) # lt_combine(SEP, PREFIX-LIST, INFIX, SUFFIX1, [SUFFIX2...]) # ---------------------------------------------------------- # Produce a SEP delimited list of all paired combinations of elements of # PREFIX-LIST with SUFFIX1 through SUFFIXn. Each element of the list # has the form PREFIXmINFIXSUFFIXn. # Needed until we can rely on m4_combine added in Autoconf 2.62. m4_define([lt_combine], [m4_if(m4_eval([$# > 3]), [1], [m4_pushdef([_Lt_sep], [m4_define([_Lt_sep], m4_defn([lt_car]))])]]dnl [[m4_foreach([_Lt_prefix], [$2], [m4_foreach([_Lt_suffix], ]m4_dquote(m4_dquote(m4_shift(m4_shift(m4_shift($@)))))[, [_Lt_sep([$1])[]m4_defn([_Lt_prefix])[$3]m4_defn([_Lt_suffix])])])])]) # lt_if_append_uniq(MACRO-NAME, VARNAME, [SEPARATOR], [UNIQ], [NOT-UNIQ]) # ----------------------------------------------------------------------- # Iff MACRO-NAME does not yet contain VARNAME, then append it (delimited # by SEPARATOR if supplied) and expand UNIQ, else NOT-UNIQ. m4_define([lt_if_append_uniq], [m4_ifdef([$1], [m4_if(m4_index([$3]m4_defn([$1])[$3], [$3$2$3]), [-1], [lt_append([$1], [$2], [$3])$4], [$5])], [lt_append([$1], [$2], [$3])$4])]) # lt_dict_add(DICT, KEY, VALUE) # ----------------------------- m4_define([lt_dict_add], [m4_define([$1($2)], [$3])]) # lt_dict_add_subkey(DICT, KEY, SUBKEY, VALUE) # -------------------------------------------- m4_define([lt_dict_add_subkey], [m4_define([$1($2:$3)], [$4])]) # lt_dict_fetch(DICT, KEY, [SUBKEY]) # ---------------------------------- m4_define([lt_dict_fetch], [m4_ifval([$3], m4_ifdef([$1($2:$3)], [m4_defn([$1($2:$3)])]), m4_ifdef([$1($2)], [m4_defn([$1($2)])]))]) # lt_if_dict_fetch(DICT, KEY, [SUBKEY], VALUE, IF-TRUE, [IF-FALSE]) # ----------------------------------------------------------------- m4_define([lt_if_dict_fetch], [m4_if(lt_dict_fetch([$1], [$2], [$3]), [$4], [$5], [$6])]) # lt_dict_filter(DICT, [SUBKEY], VALUE, [SEPARATOR], KEY, [...]) # -------------------------------------------------------------- m4_define([lt_dict_filter], [m4_if([$5], [], [], [lt_join(m4_quote(m4_default([$4], [[, ]])), lt_unquote(m4_split(m4_normalize(m4_foreach(_Lt_key, lt_car([m4_shiftn(4, $@)]), [lt_if_dict_fetch([$1], _Lt_key, [$2], [$3], [_Lt_key ])])))))])[]dnl ]) vowpal-wabbit-8.5.0.dfsg1/acinclude.d/ltversion.m4000066400000000000000000000012731323713302100217430ustar00rootroot00000000000000# ltversion.m4 -- version numbers -*- Autoconf -*- # # Copyright (C) 2004, 2011-2015 Free Software Foundation, Inc. # Written by Scott James Remnant, 2004 # # This file is free software; the Free Software Foundation gives # unlimited permission to copy and/or distribute it, with or without # modifications, as long as this notice is preserved. # @configure_input@ # serial 4179 ltversion.m4 # This file is part of GNU Libtool m4_define([LT_PACKAGE_VERSION], [2.4.6]) m4_define([LT_PACKAGE_REVISION], [2.4.6]) AC_DEFUN([LTVERSION_VERSION], [macro_version='2.4.6' macro_revision='2.4.6' _LT_DECL(, macro_version, 0, [Which release of libtool.m4 was used?]) _LT_DECL(, macro_revision, 0) ]) vowpal-wabbit-8.5.0.dfsg1/acinclude.d/lt~obsolete.m4000066400000000000000000000137741323713302100223010ustar00rootroot00000000000000# lt~obsolete.m4 -- aclocal satisfying obsolete definitions. -*-Autoconf-*- # # Copyright (C) 2004-2005, 2007, 2009, 2011-2015 Free Software # Foundation, Inc. # Written by Scott James Remnant, 2004. # # This file is free software; the Free Software Foundation gives # unlimited permission to copy and/or distribute it, with or without # modifications, as long as this notice is preserved. # serial 5 lt~obsolete.m4 # These exist entirely to fool aclocal when bootstrapping libtool. # # In the past libtool.m4 has provided macros via AC_DEFUN (or AU_DEFUN), # which have later been changed to m4_define as they aren't part of the # exported API, or moved to Autoconf or Automake where they belong. # # The trouble is, aclocal is a bit thick. It'll see the old AC_DEFUN # in /usr/share/aclocal/libtool.m4 and remember it, then when it sees us # using a macro with the same name in our local m4/libtool.m4 it'll # pull the old libtool.m4 in (it doesn't see our shiny new m4_define # and doesn't know about Autoconf macros at all.) # # So we provide this file, which has a silly filename so it's always # included after everything else. This provides aclocal with the # AC_DEFUNs it wants, but when m4 processes it, it doesn't do anything # because those macros already exist, or will be overwritten later. # We use AC_DEFUN over AU_DEFUN for compatibility with aclocal-1.6. # # Anytime we withdraw an AC_DEFUN or AU_DEFUN, remember to add it here. # Yes, that means every name once taken will need to remain here until # we give up compatibility with versions before 1.7, at which point # we need to keep only those names which we still refer to. # This is to help aclocal find these macros, as it can't see m4_define. AC_DEFUN([LTOBSOLETE_VERSION], [m4_if([1])]) m4_ifndef([AC_LIBTOOL_LINKER_OPTION], [AC_DEFUN([AC_LIBTOOL_LINKER_OPTION])]) m4_ifndef([AC_PROG_EGREP], [AC_DEFUN([AC_PROG_EGREP])]) m4_ifndef([_LT_AC_PROG_ECHO_BACKSLASH], [AC_DEFUN([_LT_AC_PROG_ECHO_BACKSLASH])]) m4_ifndef([_LT_AC_SHELL_INIT], [AC_DEFUN([_LT_AC_SHELL_INIT])]) m4_ifndef([_LT_AC_SYS_LIBPATH_AIX], [AC_DEFUN([_LT_AC_SYS_LIBPATH_AIX])]) m4_ifndef([_LT_PROG_LTMAIN], [AC_DEFUN([_LT_PROG_LTMAIN])]) m4_ifndef([_LT_AC_TAGVAR], [AC_DEFUN([_LT_AC_TAGVAR])]) m4_ifndef([AC_LTDL_ENABLE_INSTALL], [AC_DEFUN([AC_LTDL_ENABLE_INSTALL])]) m4_ifndef([AC_LTDL_PREOPEN], [AC_DEFUN([AC_LTDL_PREOPEN])]) m4_ifndef([_LT_AC_SYS_COMPILER], [AC_DEFUN([_LT_AC_SYS_COMPILER])]) m4_ifndef([_LT_AC_LOCK], [AC_DEFUN([_LT_AC_LOCK])]) m4_ifndef([AC_LIBTOOL_SYS_OLD_ARCHIVE], [AC_DEFUN([AC_LIBTOOL_SYS_OLD_ARCHIVE])]) m4_ifndef([_LT_AC_TRY_DLOPEN_SELF], [AC_DEFUN([_LT_AC_TRY_DLOPEN_SELF])]) m4_ifndef([AC_LIBTOOL_PROG_CC_C_O], [AC_DEFUN([AC_LIBTOOL_PROG_CC_C_O])]) m4_ifndef([AC_LIBTOOL_SYS_HARD_LINK_LOCKS], [AC_DEFUN([AC_LIBTOOL_SYS_HARD_LINK_LOCKS])]) m4_ifndef([AC_LIBTOOL_OBJDIR], [AC_DEFUN([AC_LIBTOOL_OBJDIR])]) m4_ifndef([AC_LTDL_OBJDIR], [AC_DEFUN([AC_LTDL_OBJDIR])]) m4_ifndef([AC_LIBTOOL_PROG_LD_HARDCODE_LIBPATH], [AC_DEFUN([AC_LIBTOOL_PROG_LD_HARDCODE_LIBPATH])]) m4_ifndef([AC_LIBTOOL_SYS_LIB_STRIP], [AC_DEFUN([AC_LIBTOOL_SYS_LIB_STRIP])]) m4_ifndef([AC_PATH_MAGIC], [AC_DEFUN([AC_PATH_MAGIC])]) m4_ifndef([AC_PROG_LD_GNU], [AC_DEFUN([AC_PROG_LD_GNU])]) m4_ifndef([AC_PROG_LD_RELOAD_FLAG], [AC_DEFUN([AC_PROG_LD_RELOAD_FLAG])]) m4_ifndef([AC_DEPLIBS_CHECK_METHOD], [AC_DEFUN([AC_DEPLIBS_CHECK_METHOD])]) m4_ifndef([AC_LIBTOOL_PROG_COMPILER_NO_RTTI], [AC_DEFUN([AC_LIBTOOL_PROG_COMPILER_NO_RTTI])]) m4_ifndef([AC_LIBTOOL_SYS_GLOBAL_SYMBOL_PIPE], [AC_DEFUN([AC_LIBTOOL_SYS_GLOBAL_SYMBOL_PIPE])]) m4_ifndef([AC_LIBTOOL_PROG_COMPILER_PIC], [AC_DEFUN([AC_LIBTOOL_PROG_COMPILER_PIC])]) m4_ifndef([AC_LIBTOOL_PROG_LD_SHLIBS], [AC_DEFUN([AC_LIBTOOL_PROG_LD_SHLIBS])]) m4_ifndef([AC_LIBTOOL_POSTDEP_PREDEP], [AC_DEFUN([AC_LIBTOOL_POSTDEP_PREDEP])]) m4_ifndef([LT_AC_PROG_EGREP], [AC_DEFUN([LT_AC_PROG_EGREP])]) m4_ifndef([LT_AC_PROG_SED], [AC_DEFUN([LT_AC_PROG_SED])]) m4_ifndef([_LT_CC_BASENAME], [AC_DEFUN([_LT_CC_BASENAME])]) m4_ifndef([_LT_COMPILER_BOILERPLATE], [AC_DEFUN([_LT_COMPILER_BOILERPLATE])]) m4_ifndef([_LT_LINKER_BOILERPLATE], [AC_DEFUN([_LT_LINKER_BOILERPLATE])]) m4_ifndef([_AC_PROG_LIBTOOL], [AC_DEFUN([_AC_PROG_LIBTOOL])]) m4_ifndef([AC_LIBTOOL_SETUP], [AC_DEFUN([AC_LIBTOOL_SETUP])]) m4_ifndef([_LT_AC_CHECK_DLFCN], [AC_DEFUN([_LT_AC_CHECK_DLFCN])]) m4_ifndef([AC_LIBTOOL_SYS_DYNAMIC_LINKER], [AC_DEFUN([AC_LIBTOOL_SYS_DYNAMIC_LINKER])]) m4_ifndef([_LT_AC_TAGCONFIG], [AC_DEFUN([_LT_AC_TAGCONFIG])]) m4_ifndef([AC_DISABLE_FAST_INSTALL], [AC_DEFUN([AC_DISABLE_FAST_INSTALL])]) m4_ifndef([_LT_AC_LANG_CXX], [AC_DEFUN([_LT_AC_LANG_CXX])]) m4_ifndef([_LT_AC_LANG_F77], [AC_DEFUN([_LT_AC_LANG_F77])]) m4_ifndef([_LT_AC_LANG_GCJ], [AC_DEFUN([_LT_AC_LANG_GCJ])]) m4_ifndef([AC_LIBTOOL_LANG_C_CONFIG], [AC_DEFUN([AC_LIBTOOL_LANG_C_CONFIG])]) m4_ifndef([_LT_AC_LANG_C_CONFIG], [AC_DEFUN([_LT_AC_LANG_C_CONFIG])]) m4_ifndef([AC_LIBTOOL_LANG_CXX_CONFIG], [AC_DEFUN([AC_LIBTOOL_LANG_CXX_CONFIG])]) m4_ifndef([_LT_AC_LANG_CXX_CONFIG], [AC_DEFUN([_LT_AC_LANG_CXX_CONFIG])]) m4_ifndef([AC_LIBTOOL_LANG_F77_CONFIG], [AC_DEFUN([AC_LIBTOOL_LANG_F77_CONFIG])]) m4_ifndef([_LT_AC_LANG_F77_CONFIG], [AC_DEFUN([_LT_AC_LANG_F77_CONFIG])]) m4_ifndef([AC_LIBTOOL_LANG_GCJ_CONFIG], [AC_DEFUN([AC_LIBTOOL_LANG_GCJ_CONFIG])]) m4_ifndef([_LT_AC_LANG_GCJ_CONFIG], [AC_DEFUN([_LT_AC_LANG_GCJ_CONFIG])]) m4_ifndef([AC_LIBTOOL_LANG_RC_CONFIG], [AC_DEFUN([AC_LIBTOOL_LANG_RC_CONFIG])]) m4_ifndef([_LT_AC_LANG_RC_CONFIG], [AC_DEFUN([_LT_AC_LANG_RC_CONFIG])]) m4_ifndef([AC_LIBTOOL_CONFIG], [AC_DEFUN([AC_LIBTOOL_CONFIG])]) m4_ifndef([_LT_AC_FILE_LTDLL_C], [AC_DEFUN([_LT_AC_FILE_LTDLL_C])]) m4_ifndef([_LT_REQUIRED_DARWIN_CHECKS], [AC_DEFUN([_LT_REQUIRED_DARWIN_CHECKS])]) m4_ifndef([_LT_AC_PROG_CXXCPP], [AC_DEFUN([_LT_AC_PROG_CXXCPP])]) m4_ifndef([_LT_PREPARE_SED_QUOTE_VARS], [AC_DEFUN([_LT_PREPARE_SED_QUOTE_VARS])]) m4_ifndef([_LT_PROG_ECHO_BACKSLASH], [AC_DEFUN([_LT_PROG_ECHO_BACKSLASH])]) m4_ifndef([_LT_PROG_F77], [AC_DEFUN([_LT_PROG_F77])]) m4_ifndef([_LT_PROG_FC], [AC_DEFUN([_LT_PROG_FC])]) m4_ifndef([_LT_PROG_CXX], [AC_DEFUN([_LT_PROG_CXX])]) vowpal-wabbit-8.5.0.dfsg1/aclocal.m4000066400000000000000000001231101323713302100171360ustar00rootroot00000000000000# generated automatically by aclocal 1.15 -*- Autoconf -*- # Copyright (C) 1996-2014 Free Software Foundation, Inc. # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY, to the extent permitted by law; without # even the implied warranty of MERCHANTABILITY or FITNESS FOR A # PARTICULAR PURPOSE. m4_ifndef([AC_CONFIG_MACRO_DIRS], [m4_defun([_AM_CONFIG_MACRO_DIRS], [])m4_defun([AC_CONFIG_MACRO_DIRS], [_AM_CONFIG_MACRO_DIRS($@)])]) m4_ifndef([AC_AUTOCONF_VERSION], [m4_copy([m4_PACKAGE_VERSION], [AC_AUTOCONF_VERSION])])dnl m4_if(m4_defn([AC_AUTOCONF_VERSION]), [2.69],, [m4_warning([this file was generated for autoconf 2.69. You have another version of autoconf. It may work, but is not guaranteed to. If you have problems, you may need to regenerate the build system entirely. To do so, use the procedure documented by the package, typically 'autoreconf'.])]) # Copyright (C) 2002-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # AM_AUTOMAKE_VERSION(VERSION) # ---------------------------- # Automake X.Y traces this macro to ensure aclocal.m4 has been # generated from the m4 files accompanying Automake X.Y. # (This private macro should not be called outside this file.) AC_DEFUN([AM_AUTOMAKE_VERSION], [am__api_version='1.15' dnl Some users find AM_AUTOMAKE_VERSION and mistake it for a way to dnl require some minimum version. Point them to the right macro. m4_if([$1], [1.15], [], [AC_FATAL([Do not call $0, use AM_INIT_AUTOMAKE([$1]).])])dnl ]) # _AM_AUTOCONF_VERSION(VERSION) # ----------------------------- # aclocal traces this macro to find the Autoconf version. # This is a private macro too. Using m4_define simplifies # the logic in aclocal, which can simply ignore this definition. m4_define([_AM_AUTOCONF_VERSION], []) # AM_SET_CURRENT_AUTOMAKE_VERSION # ------------------------------- # Call AM_AUTOMAKE_VERSION and AM_AUTOMAKE_VERSION so they can be traced. # This function is AC_REQUIREd by AM_INIT_AUTOMAKE. AC_DEFUN([AM_SET_CURRENT_AUTOMAKE_VERSION], [AM_AUTOMAKE_VERSION([1.15])dnl m4_ifndef([AC_AUTOCONF_VERSION], [m4_copy([m4_PACKAGE_VERSION], [AC_AUTOCONF_VERSION])])dnl _AM_AUTOCONF_VERSION(m4_defn([AC_AUTOCONF_VERSION]))]) # AM_AUX_DIR_EXPAND -*- Autoconf -*- # Copyright (C) 2001-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # For projects using AC_CONFIG_AUX_DIR([foo]), Autoconf sets # $ac_aux_dir to '$srcdir/foo'. In other projects, it is set to # '$srcdir', '$srcdir/..', or '$srcdir/../..'. # # Of course, Automake must honor this variable whenever it calls a # tool from the auxiliary directory. The problem is that $srcdir (and # therefore $ac_aux_dir as well) can be either absolute or relative, # depending on how configure is run. This is pretty annoying, since # it makes $ac_aux_dir quite unusable in subdirectories: in the top # source directory, any form will work fine, but in subdirectories a # relative path needs to be adjusted first. # # $ac_aux_dir/missing # fails when called from a subdirectory if $ac_aux_dir is relative # $top_srcdir/$ac_aux_dir/missing # fails if $ac_aux_dir is absolute, # fails when called from a subdirectory in a VPATH build with # a relative $ac_aux_dir # # The reason of the latter failure is that $top_srcdir and $ac_aux_dir # are both prefixed by $srcdir. In an in-source build this is usually # harmless because $srcdir is '.', but things will broke when you # start a VPATH build or use an absolute $srcdir. # # So we could use something similar to $top_srcdir/$ac_aux_dir/missing, # iff we strip the leading $srcdir from $ac_aux_dir. That would be: # am_aux_dir='\$(top_srcdir)/'`expr "$ac_aux_dir" : "$srcdir//*\(.*\)"` # and then we would define $MISSING as # MISSING="\${SHELL} $am_aux_dir/missing" # This will work as long as MISSING is not called from configure, because # unfortunately $(top_srcdir) has no meaning in configure. # However there are other variables, like CC, which are often used in # configure, and could therefore not use this "fixed" $ac_aux_dir. # # Another solution, used here, is to always expand $ac_aux_dir to an # absolute PATH. The drawback is that using absolute paths prevent a # configured tree to be moved without reconfiguration. AC_DEFUN([AM_AUX_DIR_EXPAND], [AC_REQUIRE([AC_CONFIG_AUX_DIR_DEFAULT])dnl # Expand $ac_aux_dir to an absolute path. am_aux_dir=`cd "$ac_aux_dir" && pwd` ]) # AM_CONDITIONAL -*- Autoconf -*- # Copyright (C) 1997-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # AM_CONDITIONAL(NAME, SHELL-CONDITION) # ------------------------------------- # Define a conditional. AC_DEFUN([AM_CONDITIONAL], [AC_PREREQ([2.52])dnl m4_if([$1], [TRUE], [AC_FATAL([$0: invalid condition: $1])], [$1], [FALSE], [AC_FATAL([$0: invalid condition: $1])])dnl AC_SUBST([$1_TRUE])dnl AC_SUBST([$1_FALSE])dnl _AM_SUBST_NOTMAKE([$1_TRUE])dnl _AM_SUBST_NOTMAKE([$1_FALSE])dnl m4_define([_AM_COND_VALUE_$1], [$2])dnl if $2; then $1_TRUE= $1_FALSE='#' else $1_TRUE='#' $1_FALSE= fi AC_CONFIG_COMMANDS_PRE( [if test -z "${$1_TRUE}" && test -z "${$1_FALSE}"; then AC_MSG_ERROR([[conditional "$1" was never defined. Usually this means the macro was only invoked conditionally.]]) fi])]) # Copyright (C) 1999-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # There are a few dirty hacks below to avoid letting 'AC_PROG_CC' be # written in clear, in which case automake, when reading aclocal.m4, # will think it sees a *use*, and therefore will trigger all it's # C support machinery. Also note that it means that autoscan, seeing # CC etc. in the Makefile, will ask for an AC_PROG_CC use... # _AM_DEPENDENCIES(NAME) # ---------------------- # See how the compiler implements dependency checking. # NAME is "CC", "CXX", "OBJC", "OBJCXX", "UPC", or "GJC". # We try a few techniques and use that to set a single cache variable. # # We don't AC_REQUIRE the corresponding AC_PROG_CC since the latter was # modified to invoke _AM_DEPENDENCIES(CC); we would have a circular # dependency, and given that the user is not expected to run this macro, # just rely on AC_PROG_CC. AC_DEFUN([_AM_DEPENDENCIES], [AC_REQUIRE([AM_SET_DEPDIR])dnl AC_REQUIRE([AM_OUTPUT_DEPENDENCY_COMMANDS])dnl AC_REQUIRE([AM_MAKE_INCLUDE])dnl AC_REQUIRE([AM_DEP_TRACK])dnl m4_if([$1], [CC], [depcc="$CC" am_compiler_list=], [$1], [CXX], [depcc="$CXX" am_compiler_list=], [$1], [OBJC], [depcc="$OBJC" am_compiler_list='gcc3 gcc'], [$1], [OBJCXX], [depcc="$OBJCXX" am_compiler_list='gcc3 gcc'], [$1], [UPC], [depcc="$UPC" am_compiler_list=], [$1], [GCJ], [depcc="$GCJ" am_compiler_list='gcc3 gcc'], [depcc="$$1" am_compiler_list=]) AC_CACHE_CHECK([dependency style of $depcc], [am_cv_$1_dependencies_compiler_type], [if test -z "$AMDEP_TRUE" && test -f "$am_depcomp"; then # We make a subdir and do the tests there. Otherwise we can end up # making bogus files that we don't know about and never remove. For # instance it was reported that on HP-UX the gcc test will end up # making a dummy file named 'D' -- because '-MD' means "put the output # in D". rm -rf conftest.dir mkdir conftest.dir # Copy depcomp to subdir because otherwise we won't find it if we're # using a relative directory. cp "$am_depcomp" conftest.dir cd conftest.dir # We will build objects and dependencies in a subdirectory because # it helps to detect inapplicable dependency modes. For instance # both Tru64's cc and ICC support -MD to output dependencies as a # side effect of compilation, but ICC will put the dependencies in # the current directory while Tru64 will put them in the object # directory. mkdir sub am_cv_$1_dependencies_compiler_type=none if test "$am_compiler_list" = ""; then am_compiler_list=`sed -n ['s/^#*\([a-zA-Z0-9]*\))$/\1/p'] < ./depcomp` fi am__universal=false m4_case([$1], [CC], [case " $depcc " in #( *\ -arch\ *\ -arch\ *) am__universal=true ;; esac], [CXX], [case " $depcc " in #( *\ -arch\ *\ -arch\ *) am__universal=true ;; esac]) for depmode in $am_compiler_list; do # Setup a source with many dependencies, because some compilers # like to wrap large dependency lists on column 80 (with \), and # we should not choose a depcomp mode which is confused by this. # # We need to recreate these files for each test, as the compiler may # overwrite some of them when testing with obscure command lines. # This happens at least with the AIX C compiler. : > sub/conftest.c for i in 1 2 3 4 5 6; do echo '#include "conftst'$i'.h"' >> sub/conftest.c # Using ": > sub/conftst$i.h" creates only sub/conftst1.h with # Solaris 10 /bin/sh. echo '/* dummy */' > sub/conftst$i.h done echo "${am__include} ${am__quote}sub/conftest.Po${am__quote}" > confmf # We check with '-c' and '-o' for the sake of the "dashmstdout" # mode. It turns out that the SunPro C++ compiler does not properly # handle '-M -o', and we need to detect this. Also, some Intel # versions had trouble with output in subdirs. am__obj=sub/conftest.${OBJEXT-o} am__minus_obj="-o $am__obj" case $depmode in gcc) # This depmode causes a compiler race in universal mode. test "$am__universal" = false || continue ;; nosideeffect) # After this tag, mechanisms are not by side-effect, so they'll # only be used when explicitly requested. if test "x$enable_dependency_tracking" = xyes; then continue else break fi ;; msvc7 | msvc7msys | msvisualcpp | msvcmsys) # This compiler won't grok '-c -o', but also, the minuso test has # not run yet. These depmodes are late enough in the game, and # so weak that their functioning should not be impacted. am__obj=conftest.${OBJEXT-o} am__minus_obj= ;; none) break ;; esac if depmode=$depmode \ source=sub/conftest.c object=$am__obj \ depfile=sub/conftest.Po tmpdepfile=sub/conftest.TPo \ $SHELL ./depcomp $depcc -c $am__minus_obj sub/conftest.c \ >/dev/null 2>conftest.err && grep sub/conftst1.h sub/conftest.Po > /dev/null 2>&1 && grep sub/conftst6.h sub/conftest.Po > /dev/null 2>&1 && grep $am__obj sub/conftest.Po > /dev/null 2>&1 && ${MAKE-make} -s -f confmf > /dev/null 2>&1; then # icc doesn't choke on unknown options, it will just issue warnings # or remarks (even with -Werror). So we grep stderr for any message # that says an option was ignored or not supported. # When given -MP, icc 7.0 and 7.1 complain thusly: # icc: Command line warning: ignoring option '-M'; no argument required # The diagnosis changed in icc 8.0: # icc: Command line remark: option '-MP' not supported if (grep 'ignoring option' conftest.err || grep 'not supported' conftest.err) >/dev/null 2>&1; then :; else am_cv_$1_dependencies_compiler_type=$depmode break fi fi done cd .. rm -rf conftest.dir else am_cv_$1_dependencies_compiler_type=none fi ]) AC_SUBST([$1DEPMODE], [depmode=$am_cv_$1_dependencies_compiler_type]) AM_CONDITIONAL([am__fastdep$1], [ test "x$enable_dependency_tracking" != xno \ && test "$am_cv_$1_dependencies_compiler_type" = gcc3]) ]) # AM_SET_DEPDIR # ------------- # Choose a directory name for dependency files. # This macro is AC_REQUIREd in _AM_DEPENDENCIES. AC_DEFUN([AM_SET_DEPDIR], [AC_REQUIRE([AM_SET_LEADING_DOT])dnl AC_SUBST([DEPDIR], ["${am__leading_dot}deps"])dnl ]) # AM_DEP_TRACK # ------------ AC_DEFUN([AM_DEP_TRACK], [AC_ARG_ENABLE([dependency-tracking], [dnl AS_HELP_STRING( [--enable-dependency-tracking], [do not reject slow dependency extractors]) AS_HELP_STRING( [--disable-dependency-tracking], [speeds up one-time build])]) if test "x$enable_dependency_tracking" != xno; then am_depcomp="$ac_aux_dir/depcomp" AMDEPBACKSLASH='\' am__nodep='_no' fi AM_CONDITIONAL([AMDEP], [test "x$enable_dependency_tracking" != xno]) AC_SUBST([AMDEPBACKSLASH])dnl _AM_SUBST_NOTMAKE([AMDEPBACKSLASH])dnl AC_SUBST([am__nodep])dnl _AM_SUBST_NOTMAKE([am__nodep])dnl ]) # Generate code to set up dependency tracking. -*- Autoconf -*- # Copyright (C) 1999-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # _AM_OUTPUT_DEPENDENCY_COMMANDS # ------------------------------ AC_DEFUN([_AM_OUTPUT_DEPENDENCY_COMMANDS], [{ # Older Autoconf quotes --file arguments for eval, but not when files # are listed without --file. Let's play safe and only enable the eval # if we detect the quoting. case $CONFIG_FILES in *\'*) eval set x "$CONFIG_FILES" ;; *) set x $CONFIG_FILES ;; esac shift for mf do # Strip MF so we end up with the name of the file. mf=`echo "$mf" | sed -e 's/:.*$//'` # Check whether this is an Automake generated Makefile or not. # We used to match only the files named 'Makefile.in', but # some people rename them; so instead we look at the file content. # Grep'ing the first line is not enough: some people post-process # each Makefile.in and add a new line on top of each file to say so. # Grep'ing the whole file is not good either: AIX grep has a line # limit of 2048, but all sed's we know have understand at least 4000. if sed -n 's,^#.*generated by automake.*,X,p' "$mf" | grep X >/dev/null 2>&1; then dirpart=`AS_DIRNAME("$mf")` else continue fi # Extract the definition of DEPDIR, am__include, and am__quote # from the Makefile without running 'make'. DEPDIR=`sed -n 's/^DEPDIR = //p' < "$mf"` test -z "$DEPDIR" && continue am__include=`sed -n 's/^am__include = //p' < "$mf"` test -z "$am__include" && continue am__quote=`sed -n 's/^am__quote = //p' < "$mf"` # Find all dependency output files, they are included files with # $(DEPDIR) in their names. We invoke sed twice because it is the # simplest approach to changing $(DEPDIR) to its actual value in the # expansion. for file in `sed -n " s/^$am__include $am__quote\(.*(DEPDIR).*\)$am__quote"'$/\1/p' <"$mf" | \ sed -e 's/\$(DEPDIR)/'"$DEPDIR"'/g'`; do # Make sure the directory exists. test -f "$dirpart/$file" && continue fdir=`AS_DIRNAME(["$file"])` AS_MKDIR_P([$dirpart/$fdir]) # echo "creating $dirpart/$file" echo '# dummy' > "$dirpart/$file" done done } ])# _AM_OUTPUT_DEPENDENCY_COMMANDS # AM_OUTPUT_DEPENDENCY_COMMANDS # ----------------------------- # This macro should only be invoked once -- use via AC_REQUIRE. # # This code is only required when automatic dependency tracking # is enabled. FIXME. This creates each '.P' file that we will # need in order to bootstrap the dependency handling code. AC_DEFUN([AM_OUTPUT_DEPENDENCY_COMMANDS], [AC_CONFIG_COMMANDS([depfiles], [test x"$AMDEP_TRUE" != x"" || _AM_OUTPUT_DEPENDENCY_COMMANDS], [AMDEP_TRUE="$AMDEP_TRUE" ac_aux_dir="$ac_aux_dir"]) ]) # Do all the work for Automake. -*- Autoconf -*- # Copyright (C) 1996-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # This macro actually does too much. Some checks are only needed if # your package does certain things. But this isn't really a big deal. dnl Redefine AC_PROG_CC to automatically invoke _AM_PROG_CC_C_O. m4_define([AC_PROG_CC], m4_defn([AC_PROG_CC]) [_AM_PROG_CC_C_O ]) # AM_INIT_AUTOMAKE(PACKAGE, VERSION, [NO-DEFINE]) # AM_INIT_AUTOMAKE([OPTIONS]) # ----------------------------------------------- # The call with PACKAGE and VERSION arguments is the old style # call (pre autoconf-2.50), which is being phased out. PACKAGE # and VERSION should now be passed to AC_INIT and removed from # the call to AM_INIT_AUTOMAKE. # We support both call styles for the transition. After # the next Automake release, Autoconf can make the AC_INIT # arguments mandatory, and then we can depend on a new Autoconf # release and drop the old call support. AC_DEFUN([AM_INIT_AUTOMAKE], [AC_PREREQ([2.65])dnl dnl Autoconf wants to disallow AM_ names. We explicitly allow dnl the ones we care about. m4_pattern_allow([^AM_[A-Z]+FLAGS$])dnl AC_REQUIRE([AM_SET_CURRENT_AUTOMAKE_VERSION])dnl AC_REQUIRE([AC_PROG_INSTALL])dnl if test "`cd $srcdir && pwd`" != "`pwd`"; then # Use -I$(srcdir) only when $(srcdir) != ., so that make's output # is not polluted with repeated "-I." AC_SUBST([am__isrc], [' -I$(srcdir)'])_AM_SUBST_NOTMAKE([am__isrc])dnl # test to see if srcdir already configured if test -f $srcdir/config.status; then AC_MSG_ERROR([source directory already configured; run "make distclean" there first]) fi fi # test whether we have cygpath if test -z "$CYGPATH_W"; then if (cygpath --version) >/dev/null 2>/dev/null; then CYGPATH_W='cygpath -w' else CYGPATH_W=echo fi fi AC_SUBST([CYGPATH_W]) # Define the identity of the package. dnl Distinguish between old-style and new-style calls. m4_ifval([$2], [AC_DIAGNOSE([obsolete], [$0: two- and three-arguments forms are deprecated.]) m4_ifval([$3], [_AM_SET_OPTION([no-define])])dnl AC_SUBST([PACKAGE], [$1])dnl AC_SUBST([VERSION], [$2])], [_AM_SET_OPTIONS([$1])dnl dnl Diagnose old-style AC_INIT with new-style AM_AUTOMAKE_INIT. m4_if( m4_ifdef([AC_PACKAGE_NAME], [ok]):m4_ifdef([AC_PACKAGE_VERSION], [ok]), [ok:ok],, [m4_fatal([AC_INIT should be called with package and version arguments])])dnl AC_SUBST([PACKAGE], ['AC_PACKAGE_TARNAME'])dnl AC_SUBST([VERSION], ['AC_PACKAGE_VERSION'])])dnl _AM_IF_OPTION([no-define],, [AC_DEFINE_UNQUOTED([PACKAGE], ["$PACKAGE"], [Name of package]) AC_DEFINE_UNQUOTED([VERSION], ["$VERSION"], [Version number of package])])dnl # Some tools Automake needs. AC_REQUIRE([AM_SANITY_CHECK])dnl AC_REQUIRE([AC_ARG_PROGRAM])dnl AM_MISSING_PROG([ACLOCAL], [aclocal-${am__api_version}]) AM_MISSING_PROG([AUTOCONF], [autoconf]) AM_MISSING_PROG([AUTOMAKE], [automake-${am__api_version}]) AM_MISSING_PROG([AUTOHEADER], [autoheader]) AM_MISSING_PROG([MAKEINFO], [makeinfo]) AC_REQUIRE([AM_PROG_INSTALL_SH])dnl AC_REQUIRE([AM_PROG_INSTALL_STRIP])dnl AC_REQUIRE([AC_PROG_MKDIR_P])dnl # For better backward compatibility. To be removed once Automake 1.9.x # dies out for good. For more background, see: # # AC_SUBST([mkdir_p], ['$(MKDIR_P)']) # We need awk for the "check" target (and possibly the TAP driver). The # system "awk" is bad on some platforms. AC_REQUIRE([AC_PROG_AWK])dnl AC_REQUIRE([AC_PROG_MAKE_SET])dnl AC_REQUIRE([AM_SET_LEADING_DOT])dnl _AM_IF_OPTION([tar-ustar], [_AM_PROG_TAR([ustar])], [_AM_IF_OPTION([tar-pax], [_AM_PROG_TAR([pax])], [_AM_PROG_TAR([v7])])]) _AM_IF_OPTION([no-dependencies],, [AC_PROVIDE_IFELSE([AC_PROG_CC], [_AM_DEPENDENCIES([CC])], [m4_define([AC_PROG_CC], m4_defn([AC_PROG_CC])[_AM_DEPENDENCIES([CC])])])dnl AC_PROVIDE_IFELSE([AC_PROG_CXX], [_AM_DEPENDENCIES([CXX])], [m4_define([AC_PROG_CXX], m4_defn([AC_PROG_CXX])[_AM_DEPENDENCIES([CXX])])])dnl AC_PROVIDE_IFELSE([AC_PROG_OBJC], [_AM_DEPENDENCIES([OBJC])], [m4_define([AC_PROG_OBJC], m4_defn([AC_PROG_OBJC])[_AM_DEPENDENCIES([OBJC])])])dnl AC_PROVIDE_IFELSE([AC_PROG_OBJCXX], [_AM_DEPENDENCIES([OBJCXX])], [m4_define([AC_PROG_OBJCXX], m4_defn([AC_PROG_OBJCXX])[_AM_DEPENDENCIES([OBJCXX])])])dnl ]) AC_REQUIRE([AM_SILENT_RULES])dnl dnl The testsuite driver may need to know about EXEEXT, so add the dnl 'am__EXEEXT' conditional if _AM_COMPILER_EXEEXT was seen. This dnl macro is hooked onto _AC_COMPILER_EXEEXT early, see below. AC_CONFIG_COMMANDS_PRE(dnl [m4_provide_if([_AM_COMPILER_EXEEXT], [AM_CONDITIONAL([am__EXEEXT], [test -n "$EXEEXT"])])])dnl # POSIX will say in a future version that running "rm -f" with no argument # is OK; and we want to be able to make that assumption in our Makefile # recipes. So use an aggressive probe to check that the usage we want is # actually supported "in the wild" to an acceptable degree. # See automake bug#10828. # To make any issue more visible, cause the running configure to be aborted # by default if the 'rm' program in use doesn't match our expectations; the # user can still override this though. if rm -f && rm -fr && rm -rf; then : OK; else cat >&2 <<'END' Oops! Your 'rm' program seems unable to run without file operands specified on the command line, even when the '-f' option is present. This is contrary to the behaviour of most rm programs out there, and not conforming with the upcoming POSIX standard: Please tell bug-automake@gnu.org about your system, including the value of your $PATH and any error possibly output before this message. This can help us improve future automake versions. END if test x"$ACCEPT_INFERIOR_RM_PROGRAM" = x"yes"; then echo 'Configuration will proceed anyway, since you have set the' >&2 echo 'ACCEPT_INFERIOR_RM_PROGRAM variable to "yes"' >&2 echo >&2 else cat >&2 <<'END' Aborting the configuration process, to ensure you take notice of the issue. You can download and install GNU coreutils to get an 'rm' implementation that behaves properly: . If you want to complete the configuration process using your problematic 'rm' anyway, export the environment variable ACCEPT_INFERIOR_RM_PROGRAM to "yes", and re-run configure. END AC_MSG_ERROR([Your 'rm' program is bad, sorry.]) fi fi dnl The trailing newline in this macro's definition is deliberate, for dnl backward compatibility and to allow trailing 'dnl'-style comments dnl after the AM_INIT_AUTOMAKE invocation. See automake bug#16841. ]) dnl Hook into '_AC_COMPILER_EXEEXT' early to learn its expansion. Do not dnl add the conditional right here, as _AC_COMPILER_EXEEXT may be further dnl mangled by Autoconf and run in a shell conditional statement. m4_define([_AC_COMPILER_EXEEXT], m4_defn([_AC_COMPILER_EXEEXT])[m4_provide([_AM_COMPILER_EXEEXT])]) # When config.status generates a header, we must update the stamp-h file. # This file resides in the same directory as the config header # that is generated. The stamp files are numbered to have different names. # Autoconf calls _AC_AM_CONFIG_HEADER_HOOK (when defined) in the # loop where config.status creates the headers, so we can generate # our stamp files there. AC_DEFUN([_AC_AM_CONFIG_HEADER_HOOK], [# Compute $1's index in $config_headers. _am_arg=$1 _am_stamp_count=1 for _am_header in $config_headers :; do case $_am_header in $_am_arg | $_am_arg:* ) break ;; * ) _am_stamp_count=`expr $_am_stamp_count + 1` ;; esac done echo "timestamp for $_am_arg" >`AS_DIRNAME(["$_am_arg"])`/stamp-h[]$_am_stamp_count]) # Copyright (C) 2001-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # AM_PROG_INSTALL_SH # ------------------ # Define $install_sh. AC_DEFUN([AM_PROG_INSTALL_SH], [AC_REQUIRE([AM_AUX_DIR_EXPAND])dnl if test x"${install_sh+set}" != xset; then case $am_aux_dir in *\ * | *\ *) install_sh="\${SHELL} '$am_aux_dir/install-sh'" ;; *) install_sh="\${SHELL} $am_aux_dir/install-sh" esac fi AC_SUBST([install_sh])]) # Copyright (C) 2003-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # Check whether the underlying file-system supports filenames # with a leading dot. For instance MS-DOS doesn't. AC_DEFUN([AM_SET_LEADING_DOT], [rm -rf .tst 2>/dev/null mkdir .tst 2>/dev/null if test -d .tst; then am__leading_dot=. else am__leading_dot=_ fi rmdir .tst 2>/dev/null AC_SUBST([am__leading_dot])]) # Check to see how 'make' treats includes. -*- Autoconf -*- # Copyright (C) 2001-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # AM_MAKE_INCLUDE() # ----------------- # Check to see how make treats includes. AC_DEFUN([AM_MAKE_INCLUDE], [am_make=${MAKE-make} cat > confinc << 'END' am__doit: @echo this is the am__doit target .PHONY: am__doit END # If we don't find an include directive, just comment out the code. AC_MSG_CHECKING([for style of include used by $am_make]) am__include="#" am__quote= _am_result=none # First try GNU make style include. echo "include confinc" > confmf # Ignore all kinds of additional output from 'make'. case `$am_make -s -f confmf 2> /dev/null` in #( *the\ am__doit\ target*) am__include=include am__quote= _am_result=GNU ;; esac # Now try BSD make style include. if test "$am__include" = "#"; then echo '.include "confinc"' > confmf case `$am_make -s -f confmf 2> /dev/null` in #( *the\ am__doit\ target*) am__include=.include am__quote="\"" _am_result=BSD ;; esac fi AC_SUBST([am__include]) AC_SUBST([am__quote]) AC_MSG_RESULT([$_am_result]) rm -f confinc confmf ]) # Fake the existence of programs that GNU maintainers use. -*- Autoconf -*- # Copyright (C) 1997-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # AM_MISSING_PROG(NAME, PROGRAM) # ------------------------------ AC_DEFUN([AM_MISSING_PROG], [AC_REQUIRE([AM_MISSING_HAS_RUN]) $1=${$1-"${am_missing_run}$2"} AC_SUBST($1)]) # AM_MISSING_HAS_RUN # ------------------ # Define MISSING if not defined so far and test if it is modern enough. # If it is, set am_missing_run to use it, otherwise, to nothing. AC_DEFUN([AM_MISSING_HAS_RUN], [AC_REQUIRE([AM_AUX_DIR_EXPAND])dnl AC_REQUIRE_AUX_FILE([missing])dnl if test x"${MISSING+set}" != xset; then case $am_aux_dir in *\ * | *\ *) MISSING="\${SHELL} \"$am_aux_dir/missing\"" ;; *) MISSING="\${SHELL} $am_aux_dir/missing" ;; esac fi # Use eval to expand $SHELL if eval "$MISSING --is-lightweight"; then am_missing_run="$MISSING " else am_missing_run= AC_MSG_WARN(['missing' script is too old or missing]) fi ]) # Helper functions for option handling. -*- Autoconf -*- # Copyright (C) 2001-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # _AM_MANGLE_OPTION(NAME) # ----------------------- AC_DEFUN([_AM_MANGLE_OPTION], [[_AM_OPTION_]m4_bpatsubst($1, [[^a-zA-Z0-9_]], [_])]) # _AM_SET_OPTION(NAME) # -------------------- # Set option NAME. Presently that only means defining a flag for this option. AC_DEFUN([_AM_SET_OPTION], [m4_define(_AM_MANGLE_OPTION([$1]), [1])]) # _AM_SET_OPTIONS(OPTIONS) # ------------------------ # OPTIONS is a space-separated list of Automake options. AC_DEFUN([_AM_SET_OPTIONS], [m4_foreach_w([_AM_Option], [$1], [_AM_SET_OPTION(_AM_Option)])]) # _AM_IF_OPTION(OPTION, IF-SET, [IF-NOT-SET]) # ------------------------------------------- # Execute IF-SET if OPTION is set, IF-NOT-SET otherwise. AC_DEFUN([_AM_IF_OPTION], [m4_ifset(_AM_MANGLE_OPTION([$1]), [$2], [$3])]) # Copyright (C) 1999-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # _AM_PROG_CC_C_O # --------------- # Like AC_PROG_CC_C_O, but changed for automake. We rewrite AC_PROG_CC # to automatically call this. AC_DEFUN([_AM_PROG_CC_C_O], [AC_REQUIRE([AM_AUX_DIR_EXPAND])dnl AC_REQUIRE_AUX_FILE([compile])dnl AC_LANG_PUSH([C])dnl AC_CACHE_CHECK( [whether $CC understands -c and -o together], [am_cv_prog_cc_c_o], [AC_LANG_CONFTEST([AC_LANG_PROGRAM([])]) # Make sure it works both with $CC and with simple cc. # Following AC_PROG_CC_C_O, we do the test twice because some # compilers refuse to overwrite an existing .o file with -o, # though they will create one. am_cv_prog_cc_c_o=yes for am_i in 1 2; do if AM_RUN_LOG([$CC -c conftest.$ac_ext -o conftest2.$ac_objext]) \ && test -f conftest2.$ac_objext; then : OK else am_cv_prog_cc_c_o=no break fi done rm -f core conftest* unset am_i]) if test "$am_cv_prog_cc_c_o" != yes; then # Losing compiler, so override with the script. # FIXME: It is wrong to rewrite CC. # But if we don't then we get into trouble of one sort or another. # A longer-term fix would be to have automake use am__CC in this case, # and then we could set am__CC="\$(top_srcdir)/compile \$(CC)" CC="$am_aux_dir/compile $CC" fi AC_LANG_POP([C])]) # For backward compatibility. AC_DEFUN_ONCE([AM_PROG_CC_C_O], [AC_REQUIRE([AC_PROG_CC])]) # Copyright (C) 2001-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # AM_RUN_LOG(COMMAND) # ------------------- # Run COMMAND, save the exit status in ac_status, and log it. # (This has been adapted from Autoconf's _AC_RUN_LOG macro.) AC_DEFUN([AM_RUN_LOG], [{ echo "$as_me:$LINENO: $1" >&AS_MESSAGE_LOG_FD ($1) >&AS_MESSAGE_LOG_FD 2>&AS_MESSAGE_LOG_FD ac_status=$? echo "$as_me:$LINENO: \$? = $ac_status" >&AS_MESSAGE_LOG_FD (exit $ac_status); }]) # Check to make sure that the build environment is sane. -*- Autoconf -*- # Copyright (C) 1996-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # AM_SANITY_CHECK # --------------- AC_DEFUN([AM_SANITY_CHECK], [AC_MSG_CHECKING([whether build environment is sane]) # Reject unsafe characters in $srcdir or the absolute working directory # name. Accept space and tab only in the latter. am_lf=' ' case `pwd` in *[[\\\"\#\$\&\'\`$am_lf]]*) AC_MSG_ERROR([unsafe absolute working directory name]);; esac case $srcdir in *[[\\\"\#\$\&\'\`$am_lf\ \ ]]*) AC_MSG_ERROR([unsafe srcdir value: '$srcdir']);; esac # Do 'set' in a subshell so we don't clobber the current shell's # arguments. Must try -L first in case configure is actually a # symlink; some systems play weird games with the mod time of symlinks # (eg FreeBSD returns the mod time of the symlink's containing # directory). if ( am_has_slept=no for am_try in 1 2; do echo "timestamp, slept: $am_has_slept" > conftest.file set X `ls -Lt "$srcdir/configure" conftest.file 2> /dev/null` if test "$[*]" = "X"; then # -L didn't work. set X `ls -t "$srcdir/configure" conftest.file` fi if test "$[*]" != "X $srcdir/configure conftest.file" \ && test "$[*]" != "X conftest.file $srcdir/configure"; then # If neither matched, then we have a broken ls. This can happen # if, for instance, CONFIG_SHELL is bash and it inherits a # broken ls alias from the environment. This has actually # happened. Such a system could not be considered "sane". AC_MSG_ERROR([ls -t appears to fail. Make sure there is not a broken alias in your environment]) fi if test "$[2]" = conftest.file || test $am_try -eq 2; then break fi # Just in case. sleep 1 am_has_slept=yes done test "$[2]" = conftest.file ) then # Ok. : else AC_MSG_ERROR([newly created file is older than distributed files! Check your system clock]) fi AC_MSG_RESULT([yes]) # If we didn't sleep, we still need to ensure time stamps of config.status and # generated files are strictly newer. am_sleep_pid= if grep 'slept: no' conftest.file >/dev/null 2>&1; then ( sleep 1 ) & am_sleep_pid=$! fi AC_CONFIG_COMMANDS_PRE( [AC_MSG_CHECKING([that generated files are newer than configure]) if test -n "$am_sleep_pid"; then # Hide warnings about reused PIDs. wait $am_sleep_pid 2>/dev/null fi AC_MSG_RESULT([done])]) rm -f conftest.file ]) # Copyright (C) 2009-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # AM_SILENT_RULES([DEFAULT]) # -------------------------- # Enable less verbose build rules; with the default set to DEFAULT # ("yes" being less verbose, "no" or empty being verbose). AC_DEFUN([AM_SILENT_RULES], [AC_ARG_ENABLE([silent-rules], [dnl AS_HELP_STRING( [--enable-silent-rules], [less verbose build output (undo: "make V=1")]) AS_HELP_STRING( [--disable-silent-rules], [verbose build output (undo: "make V=0")])dnl ]) case $enable_silent_rules in @%:@ ((( yes) AM_DEFAULT_VERBOSITY=0;; no) AM_DEFAULT_VERBOSITY=1;; *) AM_DEFAULT_VERBOSITY=m4_if([$1], [yes], [0], [1]);; esac dnl dnl A few 'make' implementations (e.g., NonStop OS and NextStep) dnl do not support nested variable expansions. dnl See automake bug#9928 and bug#10237. am_make=${MAKE-make} AC_CACHE_CHECK([whether $am_make supports nested variables], [am_cv_make_support_nested_variables], [if AS_ECHO([['TRUE=$(BAR$(V)) BAR0=false BAR1=true V=1 am__doit: @$(TRUE) .PHONY: am__doit']]) | $am_make -f - >/dev/null 2>&1; then am_cv_make_support_nested_variables=yes else am_cv_make_support_nested_variables=no fi]) if test $am_cv_make_support_nested_variables = yes; then dnl Using '$V' instead of '$(V)' breaks IRIX make. AM_V='$(V)' AM_DEFAULT_V='$(AM_DEFAULT_VERBOSITY)' else AM_V=$AM_DEFAULT_VERBOSITY AM_DEFAULT_V=$AM_DEFAULT_VERBOSITY fi AC_SUBST([AM_V])dnl AM_SUBST_NOTMAKE([AM_V])dnl AC_SUBST([AM_DEFAULT_V])dnl AM_SUBST_NOTMAKE([AM_DEFAULT_V])dnl AC_SUBST([AM_DEFAULT_VERBOSITY])dnl AM_BACKSLASH='\' AC_SUBST([AM_BACKSLASH])dnl _AM_SUBST_NOTMAKE([AM_BACKSLASH])dnl ]) # Copyright (C) 2001-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # AM_PROG_INSTALL_STRIP # --------------------- # One issue with vendor 'install' (even GNU) is that you can't # specify the program used to strip binaries. This is especially # annoying in cross-compiling environments, where the build's strip # is unlikely to handle the host's binaries. # Fortunately install-sh will honor a STRIPPROG variable, so we # always use install-sh in "make install-strip", and initialize # STRIPPROG with the value of the STRIP variable (set by the user). AC_DEFUN([AM_PROG_INSTALL_STRIP], [AC_REQUIRE([AM_PROG_INSTALL_SH])dnl # Installed binaries are usually stripped using 'strip' when the user # run "make install-strip". However 'strip' might not be the right # tool to use in cross-compilation environments, therefore Automake # will honor the 'STRIP' environment variable to overrule this program. dnl Don't test for $cross_compiling = yes, because it might be 'maybe'. if test "$cross_compiling" != no; then AC_CHECK_TOOL([STRIP], [strip], :) fi INSTALL_STRIP_PROGRAM="\$(install_sh) -c -s" AC_SUBST([INSTALL_STRIP_PROGRAM])]) # Copyright (C) 2006-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # _AM_SUBST_NOTMAKE(VARIABLE) # --------------------------- # Prevent Automake from outputting VARIABLE = @VARIABLE@ in Makefile.in. # This macro is traced by Automake. AC_DEFUN([_AM_SUBST_NOTMAKE]) # AM_SUBST_NOTMAKE(VARIABLE) # -------------------------- # Public sister of _AM_SUBST_NOTMAKE. AC_DEFUN([AM_SUBST_NOTMAKE], [_AM_SUBST_NOTMAKE($@)]) # Check how to create a tarball. -*- Autoconf -*- # Copyright (C) 2004-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # _AM_PROG_TAR(FORMAT) # -------------------- # Check how to create a tarball in format FORMAT. # FORMAT should be one of 'v7', 'ustar', or 'pax'. # # Substitute a variable $(am__tar) that is a command # writing to stdout a FORMAT-tarball containing the directory # $tardir. # tardir=directory && $(am__tar) > result.tar # # Substitute a variable $(am__untar) that extract such # a tarball read from stdin. # $(am__untar) < result.tar # AC_DEFUN([_AM_PROG_TAR], [# Always define AMTAR for backward compatibility. Yes, it's still used # in the wild :-( We should find a proper way to deprecate it ... AC_SUBST([AMTAR], ['$${TAR-tar}']) # We'll loop over all known methods to create a tar archive until one works. _am_tools='gnutar m4_if([$1], [ustar], [plaintar]) pax cpio none' m4_if([$1], [v7], [am__tar='$${TAR-tar} chof - "$$tardir"' am__untar='$${TAR-tar} xf -'], [m4_case([$1], [ustar], [# The POSIX 1988 'ustar' format is defined with fixed-size fields. # There is notably a 21 bits limit for the UID and the GID. In fact, # the 'pax' utility can hang on bigger UID/GID (see automake bug#8343 # and bug#13588). am_max_uid=2097151 # 2^21 - 1 am_max_gid=$am_max_uid # The $UID and $GID variables are not portable, so we need to resort # to the POSIX-mandated id(1) utility. Errors in the 'id' calls # below are definitely unexpected, so allow the users to see them # (that is, avoid stderr redirection). am_uid=`id -u || echo unknown` am_gid=`id -g || echo unknown` AC_MSG_CHECKING([whether UID '$am_uid' is supported by ustar format]) if test $am_uid -le $am_max_uid; then AC_MSG_RESULT([yes]) else AC_MSG_RESULT([no]) _am_tools=none fi AC_MSG_CHECKING([whether GID '$am_gid' is supported by ustar format]) if test $am_gid -le $am_max_gid; then AC_MSG_RESULT([yes]) else AC_MSG_RESULT([no]) _am_tools=none fi], [pax], [], [m4_fatal([Unknown tar format])]) AC_MSG_CHECKING([how to create a $1 tar archive]) # Go ahead even if we have the value already cached. We do so because we # need to set the values for the 'am__tar' and 'am__untar' variables. _am_tools=${am_cv_prog_tar_$1-$_am_tools} for _am_tool in $_am_tools; do case $_am_tool in gnutar) for _am_tar in tar gnutar gtar; do AM_RUN_LOG([$_am_tar --version]) && break done am__tar="$_am_tar --format=m4_if([$1], [pax], [posix], [$1]) -chf - "'"$$tardir"' am__tar_="$_am_tar --format=m4_if([$1], [pax], [posix], [$1]) -chf - "'"$tardir"' am__untar="$_am_tar -xf -" ;; plaintar) # Must skip GNU tar: if it does not support --format= it doesn't create # ustar tarball either. (tar --version) >/dev/null 2>&1 && continue am__tar='tar chf - "$$tardir"' am__tar_='tar chf - "$tardir"' am__untar='tar xf -' ;; pax) am__tar='pax -L -x $1 -w "$$tardir"' am__tar_='pax -L -x $1 -w "$tardir"' am__untar='pax -r' ;; cpio) am__tar='find "$$tardir" -print | cpio -o -H $1 -L' am__tar_='find "$tardir" -print | cpio -o -H $1 -L' am__untar='cpio -i -H $1 -d' ;; none) am__tar=false am__tar_=false am__untar=false ;; esac # If the value was cached, stop now. We just wanted to have am__tar # and am__untar set. test -n "${am_cv_prog_tar_$1}" && break # tar/untar a dummy directory, and stop if the command works. rm -rf conftest.dir mkdir conftest.dir echo GrepMe > conftest.dir/file AM_RUN_LOG([tardir=conftest.dir && eval $am__tar_ >conftest.tar]) rm -rf conftest.dir if test -s conftest.tar; then AM_RUN_LOG([$am__untar /dev/null 2>&1 && break fi done rm -rf conftest.dir AC_CACHE_VAL([am_cv_prog_tar_$1], [am_cv_prog_tar_$1=$_am_tool]) AC_MSG_RESULT([$am_cv_prog_tar_$1])]) AC_SUBST([am__tar]) AC_SUBST([am__untar]) ]) # _AM_PROG_TAR m4_include([acinclude.d/ax_boost_base.m4]) m4_include([acinclude.d/ax_boost_program_options.m4]) m4_include([acinclude.d/ax_check_zlib.m4]) m4_include([acinclude.d/ax_cxx_compile_stdcxx_11.m4]) m4_include([acinclude.d/ax_pthread.m4]) m4_include([acinclude.d/libtool.m4]) m4_include([acinclude.d/ltoptions.m4]) m4_include([acinclude.d/ltsugar.m4]) m4_include([acinclude.d/ltversion.m4]) m4_include([acinclude.d/lt~obsolete.m4]) vowpal-wabbit-8.5.0.dfsg1/appveyor.yml000066400000000000000000000027511323713302100176750ustar00rootroot00000000000000version: 1.0.{build} os: Windows Server 2012 R2 configuration: - Release platform: x64 shallow_clone: true clone_folder: C:\vowpal_wabbit # need to install nuget packages before Visual Studio starts to make ANTLR targets available. build_script: - cd c:\vowpal_wabbit - vowpalwabbit\.nuget\nuget install -o vowpalwabbit\packages cs\cs\packages.config - vowpalwabbit\.nuget\nuget install -o vowpalwabbit\packages cs\azure\packages.config - vowpalwabbit\.nuget\nuget install -o vowpalwabbit\packages cs\azure_worker\packages.config - vowpalwabbit\.nuget\nuget install -o vowpalwabbit\packages cs\setup_bundle\packages.config - vowpalwabbit\.nuget\nuget install -o vowpalwabbit\packages cs\unittest\packages.config - vowpalwabbit\.nuget\nuget install -o vowpalwabbit\packages python\windows27\packages.config - vowpalwabbit\.nuget\nuget install -o vowpalwabbit\packages python\windows35\packages.config - vowpalwabbit\.nuget\nuget install -o vowpalwabbit\packages vowpalwabbit\packages.config - '"C:\Program Files (x86)\MSBuild\14.0\Bin\MSBuild.exe" "C:\vowpal_wabbit\vowpalwabbit\vw.sln" /m /verbosity:normal /p:Configuration=Release;Platform=x64' test_script: - ps: >- if($env:PLATFORM -eq "x64") { cd c:\vowpal_wabbit vstest.console /Platform:x64 /inIsolation vowpalwabbit\x64\Release\cs_unittest.dll /TestCaseFilter:"TestCategory!=NotOnVSO" } on_failure: - ps: Get-ChildItem C:\Users\appveyor\AppData\Local\CrashDumps\\*.dmp | % { Push-AppveyorArtifact $_.FullName -FileName $_.Name } vowpal-wabbit-8.5.0.dfsg1/autogen.sh000077500000000000000000000020621323713302100173010ustar00rootroot00000000000000#!/bin/sh if [ -z $CXX ]; then if [ -x "`which g++`" ]; then CXX=g++ elif [ -x "`which clang++`" ]; then CXX=clang++ fi fi case $( uname -s ) in Darwin) alias vwlibtool=glibtoolize if [ -z $AC_PATH ]; then if [ -d /opt/local/share ]; then AC_PATH="/opt/local/share" else AC_PATH="/usr/local/share" fi fi ;; Linux) AC_PATH=/usr/share ldconfig="" for p in `echo ${PATH} | sed 's/:/ /g'` /sbin /usr/sbin; do if test -x ${p}/ldconfig; then ldconfig=${p}/ldconfig break fi done if test "x${ldconfig}" = x; then ldconfig=ldconfig fi LIBFILE=`${ldconfig} -p | grep program_options | tail -n 1 | cut -d '>' -f 2` echo "Boost at: $LIBFILE" BOOST_DIR_ARG="--with-boost-libdir=`dirname $LIBFILE`" echo "Using $BOOST_DIR_ARG" alias vwlibtool=libtoolize ;; *) alias vwlibtool=libtoolize ${AC_PATH:=/usr/share} ;; esac vwlibtool -f -c && aclocal -I ./acinclude.d -I $AC_PATH/aclocal && autoheader && touch README && automake -ac -Woverride && autoconf && ./configure "$@" $BOOST_DIR_ARG CXX=$CXX vowpal-wabbit-8.5.0.dfsg1/big_tests/000077500000000000000000000000001323713302100172635ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/HOWTO.use_it.txt000066400000000000000000000063011323713302100222130ustar00rootroot00000000000000Help file for using this testing framework ------------------------------------------ If you ever forget what you're currently reading, you can always see it again by typing make help To run all the tests until one fails, type make To run all the tests regardless of failures, type make -i Either way, you would be making the default target, named 'valid'. (Naming it 'test' would get confusing when you start referring to data train/test splits.) To clean up so that you can start over, type make clean To download and prepare all the relevant data at once, type make prepData To erase all the data, type make eraseData If you change some code so that the output of some tests changes, or you add some new tests, and you want to save the new output so that future outputs will be compared with it, type make expected You can work with individual tests by prepending their name to the make target, as in make 2.valid make t17.clean make myNewTest.expected Of course, you can make multiple targets together, as in make 2.clean 2.valid There are also pre-defined groups of tests that you can run, such as make regression_group If you want to create the correct outputs for a test without actually running it, type make $testName.pretend This can be useful if you want to run test X, which depends on the outputs of test Y, but you can't run test Y for some reason. The "valid" target for each test is actually a "run" followed by a "compare". Thus make 3.run 3.compare is equivalent to make 3.valid and it's possible to invoke either part without the other. If you want to use an executable other than ../vowpalwabbit/vw, you can supply it on the command line like this: make EXEC=myProgram or make ../vowpalwabbit/vw a sym-link to it. Other variables, such as ARCH, DIFF, TIME, WGET, and GREP, can also be set on the command line similarly. If you want to pass extra parameters to all tests, set the EP variable, like this: make EP="--random_seed 13" If you want to check speed in addition to accuracy, invoke with TIMING=y . Due to the many factors that can change execution speed, we do not expect speed results to be portable across machines. Therefore, "expected" speed files are not checked into the repository. If you'd like to do regression testing for speed, you should `make TIMING=y; make TIMING=y expected` on your particular machine before making any code changes. In fact, it's a good idea to then `make TIMING=y` a couple more times, and `make TIMING=y expected` again if the speed drops significantly. This can happen even on the same machine, if it's dividing attention between multiple processes. If you want to add or modify tests, read HOWTO.write_new_tests.txt . Working with data archives -------------------------- To save time and bandwidth, you can create and retrieve all the data used in all the tests at once. To create an archive of all the data, type make ARF= archive You can then stash the archive wherever you like, probably at a URL-accessible location in the cloud. Later, to retrieve the data again, you can type make URL= prepData This will cause the data to be downloaded and installed in prepared form, so that you don't need to run the preparation steps. vowpal-wabbit-8.5.0.dfsg1/big_tests/HOWTO.write_new_tests.txt000066400000000000000000000070641323713302100241570ustar00rootroot00000000000000HOWTO write new tests for the big_tests test suite ================================================== There are 3 steps in writing a new test. 1. Add code to dataSets.mk for obtaining and preparing relevant data, if it's not already there for other tests. 2. Add some variable settings to the file testDetails.mk. 3. Create the "expected" files. Note that your new tests WILL NOT EVEN RUN unless at least the 'out' and 'err' expected files exist in the right place. Read on for details. Step 1: dataSets.mk ------------------- The data will typically live under dataSets/ . Look at dataSets.mk for examples of how to add new datasets, and to see what data sets are already there. Whenever you add a new data set, don't forget to add its name to the 'allData' target at the top of dataSets.mk . Step 2: testDetails.mk ---------------------- Be sure to use := and not = to set these variables. All variables are optional. For most tests, you will want to set the variables *.params and *.inData, but it's possible to have tests without them. Here's an example of a complete test specification: 1a.inData := $(dataDir)/mnist.dir/train.prep 1a.params := --oaa 10 -d $(1a.inData) -f $(stageDir)/1a.dir/mnist.model -b 24 --adaptive --invariant --holdout_off -l 0.1 --nn 40 --passes 24 -k --compressed --cache_file $(stageDir)/1a.dir/mnist.cache The file that *.inData refers to is always treated as a prerequisite for running a test. So if the file doesn't exist, an attempt will be made to prepare it, which might also involve downloading it. By default, STDOUT will be saved in the file 'out'. STDERR will be filtered through `grep "average loss"`, and the results will be saved in the file 'err'. Other created files that should be compared to "expected" files should be captured by the *.otherOutputs variable. For example: myTest.otherOutputs := 0001_ftrl.model If you want to capture different parts of STDOUT and STDERR, you can set the variables *.STDOUT_COMPARATOR_REGEXP and *.STDERR_COMPARATOR_REGEXP, respectively. For example, to save all of STDERR without filtering, set myTest.STDERR_COMPARATOR_REGEXP := "." To use a non-standard executable (e.g. library_example), give its path to the *.exec variable, relative to the directory of the top level Makefile. For example: myTest.exec := $(TOP_MK_DIR)/../vowpalwabbit/library_example A test can have any set of targets as prerequisites, in addition to the value of *.inData, including those that merely run other tests (*.run targets) and those that check whether another test passed (*.valid targets). List inter-test dependencies like this: myTest.deps: test2.run test4.valid Unfortunatley, tracking of inter-test dependencies as above doesn't work for older versions of gnu make (< 4.0), due to bugs in those older versions. Upgrade if you can. Groups of tests are defined in testGroups.mk. You can add your tests to these groups or create new groups. Step 3: expected files ---------------------- Every test will create files named 'out' and 'err' (even if they are empty), and possibly others. When you run `make $testName.compare` (which is the 2nd part of running `make $testName.valid`) these files are compared to the expected files with the same names in the directory expected/$testName/ . The $testName.compare target WILL NOT RUN unless the expected files exist in that directory. So, the first time you create a test, you should `make $testName.run` and `make $testName.expected`. Only then will you be able to `make $testName.valid`. If you want to share you test with others, don't forget to `git add expected/$testName/`. vowpal-wabbit-8.5.0.dfsg1/big_tests/Makefile000066400000000000000000000207021323713302100207240ustar00rootroot00000000000000# Makefile for big_tests SHELL = bash # suppress built-in rules, to speed things up MAKEFLAGS+=r # disable implicit suffixes, to speed things up .SUFFIXES: # keep all intermediate files .SECONDARY: .SECONDEXPANSION: # .ONESHELL doesn't work with make < 3.82! # .ONESHELL: # comment out for debugging .SILENT: # can't compare before running; can't prep data before downloading; etc. .NOTPARALLEL: .PHONY: default all stub help valid expected pretend run startTimer clean spotless %.valid %.expected %.run %.clean %.deps %.depsWrapper %.depsStart %.depsEnd %.stub TOP_MK_DIR := $(patsubst %/,%,$(dir $(CURDIR)/$(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST)))) ifdef TIMING TIMING_SUFFIX = WithTiming MIPS_HELP := $(shell lscpu | grep -i bogomips) MIPS := $(word $(words $(MIPS_HELP)),$(MIPS_HELP)) ARCH ?= $(subst /,_,$(shell uname -o)_$(MIPS)) # can't use the architecture as directory name, cuz that produces different expected files on different platforms # stageDir := $(ARCH) TIME ?= env time -f "%U" TIME_DIFF_TOLERANCE_REL ?= 0.05 TIME_DIFF_TOLERANCE_ABS ?= 1.0 endif stageDir ?= runs dataDir ?= dataSets expDir ?= expected testCodeDir := $(TOP_MK_DIR)/testCode allTests := $(patsubst %/,%,$(shell ls $(expDir))) # allTests := 2 # the ?= allows the user to override the executable from the command line, as in # make EXEC=myProgram all # alas, $(realpath ...) doesn't seem to work here on make 3.81 for some reason EXEC ?= ../vowpalwabbit/vw ACC_DIFF_TOLERANCE_REL ?= 0.001 ACC_DIFF_TOLERANCE_ABS ?= 0.001 DIFF ?= $(testCodeDir)/floatingTolerance.pl GREP ?= grep DEFAULT_STDOUT_COMPARATOR_REGEXP ?= "." DEFAULT_STDERR_COMPARATOR_REGEXP ?= "average loss" ARF ?= vwBigtests.tz2 # you can uncomment the next line to get the data from a cache by default, but we don't recommend it URL ?= https://vowpalwabbitdata.blob.core.windows.net/bigtests/vwBigtests.tz2 all: valid ; .DEFAULT: @echo "No rules found for $@ in $(MAKEFILE_LIST)" # details of individual tests and test groups come from here include dataSets.mk include testDetails.mk include testGroups.mk # for debugging %.stub: echo $($*.deps) echo $(stageDir)/$*.dir/done echo $(addsuffix .valid,$($*.deps)) # for debugging stub: echo $(TIMING) echo $(MAKE_VERSION) echo $(ARCH) echo $(MAKEFLAGS) echo $(EXEC) echo $(TOP_MK_DIR) echo $(VPATH) help: cat $(TOP_MK_DIR)/README.txt validTargets := $(addsuffix .valid,$(allTests)) valid: $(validTargets) timingSummary @echo "test suite finished" # use this target to run without doing any diffs runTargets := $(addsuffix .run,$(allTests)) run: prepData $(runTargets) timingSummary @echo "'make run' finished" timingSummary: if [ "$(TIMING)" = "y" ] ; then \ totalTime=0 ;\ for dir in $(allTests) ; do \ timeFile=$(stageDir)/$$dir.dir/$(ARCH)/rc ;\ currTime=`cat $$timeFile` ;\ totalTime=`echo "$$totalTime + $$currTime" | bc -l` ;\ done ;\ echo "Total test time = $$totalTime" ;\ fi mcompareTargets := $(addsuffix .compare,$(allTests)) compare: $(compareTargets) @echo "'make compare' finished" expectedTargets := $(addsuffix .expected,$(allTests)) expected: $(expectedTargets) @echo "'make expected' finished" pretendTargets := $(addsuffix .pretend,$(allTests)) pretend: $(pretendTargets) @echo "'make pretend' finished" clean: -rm -r $(stageDir) @echo "'make clean' finished" %.dir/$(ARCH) %.dir: @mkdir -p $@ %.valid: %.run %.compare ; # dependency tracking doesn't work in make < 4 due to a bug in how SECONDEXPANSION is done ifeq ($(findstring 4.,$(MAKE_VERSION)),4.) %.run: %.depsWrapper $(stageDir)/%.dir/done$(TIMING_SUFFIX) ; else %.run: $(stageDir)/%.dir/done$(TIMING_SUFFIX) ; endif $(dataDir) $(stageDir): @mkdir -p $@ %.depsWrapper: %.depsStart $$(%.deps) %.depsEnd ; %.depsStart: @if [ -n "$($*.deps)" ] ; then \ echo "test $* has dependencies: $($*.deps)" ;\ fi %.depsEnd: @if [ -n "$($*.deps)" ] ; then \ echo "dependencies of test $* satisfied" ;\ fi # Don't do like on the following line, because making the dir a dependency of the file that's in it will result in the file always getting remade, cuz the dir will always have a newer timestamp. # $(stageDir)/%.dir/done: $(stageDir)/%.dir $(stageDir)/%.dir/doneWithTiming: $(EXEC) $$(%.inData) @echo "Starting to run test $*" dir=`dirname $@` ;\ mkdir -p $$dir ;\ exec=$(EXEC) ;\ if [ -n "$($*.exec)" ] ; then exec="$($*.exec)" ; fi ;\ roFile=$$dir/raw_out ;\ reFile=$$dir/raw_err ;\ oFile=$$dir/out ;\ eFile=$$dir/err ;\ mkdir -p $$dir/$(ARCH) ;\ timeFile=$$dir/$(ARCH)/rc ;\ echo "$(TIME) -o $$timeFile $$exec $($*.params) $(EP) > $$roFile 2> $$reFile" ;\ $(TIME) -o $$timeFile $$exec $($*.params) $(EP) > $$roFile 2> $$reFile ;\ uTime=`cat $$timeFile` ;\ echo "run time of test $* = $$uTime" ;\ out_compar=$(DEFAULT_STDOUT_COMPARATOR_REGEXP) ;\ if [ -n "$($*.STDOUT_COMPARATOR_REGEXP)" ] ; then out_compar="$($*.STDOUT_COMPARATOR_REGEXP)" ; fi ;\ echo "$(GREP) \"$$out_compar\" $$roFile | tee $$oFile" ;\ $(GREP) "$$out_compar" $$roFile | tee $$oFile ;\ err_compar=$(DEFAULT_STDERR_COMPARATOR_REGEXP) ;\ if [ -n "$($*.STDERR_COMPARATOR_REGEXP)" ] ; then err_compar="$($*.STDERR_COMPARATOR_REGEXP)" ; fi ;\ echo "$(GREP) \"$$err_compar\" $$reFile | tee $$eFile" ;\ $(GREP) "$$err_compar" $$reFile | tee $$eFile ;\ touch $$dir/done touch $@ @echo "Finished running test $*" $(stageDir)/%.dir/done: $(EXEC) $$(%.inData) @echo "Starting to run test $*" dir=`dirname $@` ;\ mkdir -p $$dir ;\ exec=$(EXEC) ;\ if [ -n "$($*.exec)" ] ; then exec="$($*.exec)" ; fi ;\ roFile=$$dir/raw_out ;\ reFile=$$dir/raw_err ;\ oFile=$$dir/out ;\ eFile=$$dir/err ;\ echo "$$exec $($*.params) $(EP) > $$roFile 2> $$reFile" ;\ $$exec $($*.params) $(EP) > $$roFile 2> $$reFile ;\ out_compar=$(DEFAULT_STDOUT_COMPARATOR_REGEXP) ;\ if [ -n "$($*.STDOUT_COMPARATOR_REGEXP)" ] ; then out_compar="$($*.STDOUT_COMPARATOR_REGEXP)" ; fi ;\ echo "$(GREP) \"$$out_compar\" $$roFile | tee $$oFile" ;\ $(GREP) "$$out_compar" $$roFile | tee $$oFile ;\ err_compar=$(DEFAULT_STDERR_COMPARATOR_REGEXP) ;\ if [ -n "$($*.STDERR_COMPARATOR_REGEXP)" ] ; then err_compar="$($*.STDERR_COMPARATOR_REGEXP)" ; fi ;\ echo "$(GREP) \"$$err_compar\" $$reFile | tee $$eFile" ;\ $(GREP) "$$err_compar" $$reFile | tee $$eFile ;\ touch $@ @echo "Finished running test $*" # first diff all the results, then set exit status if non-empty %.compare: $(DIFF) $(expDir)/%/out $(expDir)/%/err dir=$(stageDir)/$*.dir ;\ rm -f $$dir/diffs ;\ for FILE in out err $($*.otherOutputs) ; do \ echo "$(DIFF) $(ACC_DIFF_TOLERANCE_REL) $(ACC_DIFF_TOLERANCE_ABS) $$dir/$$FILE $(expDir)/$*/$$FILE" ;\ $(DIFF) $(ACC_DIFF_TOLERANCE_REL) $(ACC_DIFF_TOLERANCE_ABS) $$dir/$$FILE $(expDir)/$*/$$FILE |& tee -a $$dir/diffs ;\ done ;\ if [ -s $$dir/diffs ] ; then \ echo "Test $* failed due to accuracy diffs." ;\ exit 1 ;\ fi ;\ if [ "$(TIMING)" = "y" ] ; then \ timeFile=$$dir/$(ARCH)/rc ;\ uTime=`cat $$timeFile` ;\ echo "run time of test $* = $$uTime" ;\ if [ -s $(expDir)/$*/$(ARCH)/rc ] ; then (\ echo "$(DIFF) $(TIME_DIFF_TOLERANCE_REL) $(TIME_DIFF_TOLERANCE_ABS) $$timeFile $(expDir)/$*/$(ARCH)/rc" ;\ $(DIFF) $(TIME_DIFF_TOLERANCE_REL) $(TIME_DIFF_TOLERANCE_ABS) $$timeFile $(expDir)/$*/$(ARCH)/rc |& tee -a $$dir/diffs ;\ if [ -s $$dir/diffs ] ; then \ echo "Test $* failed due to speed diffs." ;\ exit 1 ;\ fi ;\ ) ;\ else (\ echo "WARNING: Missing expected timing file. You might want to create it with 'make TIMING=y expected'. Timing validation skipped." \ ) ; \ fi ;\ fi # update expected files from current output files # expected files always include out and err %.expected: dir=$(stageDir)/$*.dir ;\ mkdir -p $(expDir)/$*/ ;\ for FILE in out err $($*.otherOutputs) ; do \ (echo "copying $$dir/$$FILE to $(expDir)/$*/" ; cp $$dir/$$FILE $(expDir)/$*/ ) ;\ done ;\ if [ "$(TIMING)" = "y" ] ; then \ mkdir -p $(expDir)/$*/$(ARCH) ;\ (echo "copying $$dir/$(ARCH)/rc to $(expDir)/$*/$(ARCH)/rc" ; cp $$dir/$(ARCH)/rc $(expDir)/$*/$(ARCH)/rc ) ;\ fi # copy outputs from expected files, to pretend that the module succeeded %.pretend: $(stageDir)/%.dir $(expDir)/%/out $(expDir)/%/err dir=$(stageDir)/$*.dir ;\ for FILE in out err $($*.otherOutputs) ; do \ echo "copying $(expDir)/$*/$$FILE to $$dir/" ;\ cp $(expDir)/$*/$$FILE $$dir/ ;\ done ;\ touch $$dir/done ;\ if [ "$(TIMING)" = "y" ] ; then \ mkdir -p $$dir/$(ARCH) ;\ echo "copying $(expDir)/$*/$(ARCH)/rc $$dir/$(ARCH)/rc" ;\ cp $(expDir)/$*/$(ARCH)/rc $$dir/$(ARCH)/rc ;\ fi %.clean: -rm -rf $(stageDir)/$*.dir @echo "finished cleaning $(stageDir)/$*.dir" vowpal-wabbit-8.5.0.dfsg1/big_tests/README.txt000077700000000000000000000000001323713302100237042HOWTO.use_it.txtustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/dataSets.mk000066400000000000000000000115651323713302100213740ustar00rootroot00000000000000# Makefile for obtaining and preparing data sets allData := mnist covtype URLRep ER movielens ################ begin generic stuff ######### VPATH+=$(testCodeDir) WGET ?= wget -nv -N --no-use-server-timestamps --no-check-certificate .PHONY: getData prepData archive eraseData # archive file name can be specified on the command line as ARF= archive: cd $(dataDir) find . -name '*prep' -print0 | tar -cjhv --null -f $(ARF) -T - getData: $(dataDir) cd $(dataDir) $(WGET) $(URL) fName=`basename $(URL)` ; tar xjvmf $$fName # If a URL is specified, then simply download all the data # pre-prepped. Checked-in check-sums should be used to guard against # something missing or corrupted in the archive. allDataTargets := $(addsuffix .prep,$(allData)) ifdef URL prepData: getData else prepData: $(allDataTargets) endif @echo "finished preparing all data" %.prep: $(dataDir)/%.dir/prep ; # allDataDirs := $(addprefix $(dataDir)/,$(addsuffix .dir,$(allData))) eraseData: -rm -r $(dataDir) @echo "finished erasing all data" ################ end generic stuff ######### #OCR OCR.prep: $(dataDir)/OCR.dir/train.prep $(dataDir)/OCR.dir/test.prep ; $(dataDir)/OCR.dir/train.prep: $(dataDir)/OCR.dir/test.prep ; $(dataDir)/OCR.dir/test.prep: $(dataDir)/OCR.dir/letter.data.gz $(dataDir)/OCR.dir/letter.names dir=$(dir $@) ;\ cd $$dir ;\ $(testCodeDir)/ocr2vw.py letter.data.gz letter.names train.prep test.prep $(dataDir)/OCR.dir/letter.data.gz: dir=$(dir $@) ;\ mkdir -p $$dir ;\ cd $$dir ;\ $(WGET) http://ai.stanford.edu/~btaskar/ocr/letter.data.gz $(dataDir)/OCR.dir/letter.names: dir=$(dir $@) ;\ mkdir -p $$dir ;\ cd $$dir ;\ $(WGET) http://ai.stanford.edu/~btaskar/ocr/letter.names #movielens movielens.prep: $(dataDir)/movielens.dir/train.prep ; $(dataDir)/movielens.dir/train.prep: $(dataDir)/movielens.dir/test.prep ; cd $(dataDir)/movielens.dir/ ;\ perl -ne 'BEGIN { srand 8675309; }; \ 1; print join "\t", rand (), $$_;' \ pre.train.vw | sort -k1 | \ cut -f2- > train.prep $(dataDir)/movielens.dir/test.prep: $(dataDir)/movielens.dir/ml-1m.zip cd $(dataDir)/movielens.dir/ ;\ unzip -ou ml-1m.zip ;\ $(testCodeDir)/movielensRatings2vw.pl pre.train.vw test.prep ml-1m/ratings.dat $(dataDir)/movielens.dir/ml-1m.zip: dir=$(dir $@) ;\ mkdir -p $$dir ;\ cd $$dir ;\ $(WGET) http://files.grouplens.org/datasets/movielens/ml-1m.zip #ER ER.prep: $(dataDir)/ER.dir/train.prep $(dataDir)/ER.dir/test.prep ; $(dataDir)/ER.dir/train.prep: $(dataDir)/ER.dir/ER_train.vw cd $(dataDir)/ER.dir/ ;\ ln -sf ER_train.vw train.prep $(dataDir)/ER.dir/test.prep: $(dataDir)/ER.dir/ER_test.vw cd $(dataDir)/ER.dir/ ;\ ln -sf ER_test.vw test.prep $(dataDir)/ER.dir/ER_train.vw: $(dataDir)/ER.dir/ER_test.vw ; touch $@ $(dataDir)/ER.dir/ER_test.vw: $(dataDir)/ER.dir/er.zip cd $(dataDir)/ER.dir/ ;\ unzip -ou er.zip ;\ touch ER_test.vw $(dataDir)/ER.dir/er.zip: dir=$(dir $@) ;\ mkdir -p $$dir ;\ cd $$dir ;\ $(WGET) http://web.engr.illinois.edu/~kchang10/data/er.zip # URLRep $(dataDir)/URLRep.dir/prep: $(dataDir)/URLRep.dir/url_svmlight.tar.gz URLRep.munge.sh export testCodeDir=$(testCodeDir) ;\ cd $(dataDir)/URLRep.dir/ ;\ $(testCodeDir)/URLRep.munge.sh url_svmlight.tar.gz > prep $(dataDir)/URLRep.dir/url_svmlight.tar.gz: dir=$(dir $@) ;\ mkdir -p $$dir ;\ cd $$dir ;\ $(WGET) https://archive.ics.uci.edu/ml/machine-learning-databases/url/url_svmlight.tar.gz # COVERTYPE $(dataDir)/covtype.dir/prep: $(dataDir)/covtype.dir/covtype.data.gz covtype.munge.sh export testCodeDir=$(testCodeDir) ;\ cd $(dataDir)/covtype.dir/ ;\ $(testCodeDir)/covtype.munge.sh covtype.data.gz > prep $(dataDir)/covtype.dir/covtype.data.gz: dir=$(dir $@) ;\ mkdir -p $$dir ;\ cd $$dir ;\ $(WGET) https://archive.ics.uci.edu/ml/machine-learning-databases/covtype/covtype.data.gz ## MNIST # override implicit %.prep rule mnist.prep: $(dataDir)/mnist.dir/train.prep $(dataDir)/mnist.dir/test.prep ; $(dataDir)/mnist.dir/train.prep: mnist.extractfeatures mnist.extract-labels.pl shuffle.pl $(dataDir)/mnist.dir/train-labels-idx1-ubyte.gz $(dataDir)/mnist.dir/train-images-idx3-ubyte.gz mnist.munge.sh export testCodeDir=$(testCodeDir) ;\ cd $(dataDir)/mnist.dir/ ;\ $(testCodeDir)/mnist.munge.sh train-labels-idx1-ubyte.gz train-images-idx3-ubyte.gz \ | $(testCodeDir)/shuffle.pl > train.prep $(dataDir)/mnist.dir/test.prep: mnist.munge.sh mnist.extractfeatures mnist.extract-labels.pl $(dataDir)/mnist.dir/t10k-labels-idx1-ubyte.gz $(dataDir)/mnist.dir/t10k-images-idx3-ubyte.gz export testCodeDir=$(testCodeDir) ;\ cd $(dataDir)/mnist.dir/ ;\ $(testCodeDir)/mnist.munge.sh t10k-labels-idx1-ubyte.gz t10k-images-idx3-ubyte.gz > test.prep mnist.extractfeatures: mnist.extractfeatures.cpp cd $(testCodeDir)/ ;\ g++ -O3 -Wall $^ -o $@ $(dataDir)/mnist.dir/%.gz: dir=$(dir $@) ;\ mkdir -p $$dir ;\ cd $$dir ;\ fileName=`basename $@` ;\ $(WGET) http://yann.lecun.com/exdb/mnist/$$fileName vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/000077500000000000000000000000001323713302100210645ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/1a/000077500000000000000000000000001323713302100213655ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/1a/err000066400000000000000000000000301323713302100220710ustar00rootroot00000000000000average loss = 0.013496 vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/1a/out000066400000000000000000000000001323713302100221050ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/1b/000077500000000000000000000000001323713302100213665ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/1b/err000066400000000000000000000000301323713302100220720ustar00rootroot00000000000000average loss = 0.021500 vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/1b/out000066400000000000000000000000001323713302100221060ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/2/000077500000000000000000000000001323713302100212255ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/2/err000066400000000000000000000000301323713302100217310ustar00rootroot00000000000000average loss = 0.181223 vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/2/out000066400000000000000000000000001323713302100217450ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/3/000077500000000000000000000000001323713302100212265ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/3/err000066400000000000000000000000301323713302100217320ustar00rootroot00000000000000average loss = 0.009036 vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/3/out000066400000000000000000000000001323713302100217460ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/4a/000077500000000000000000000000001323713302100213705ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/4a/err000066400000000000000000000000301323713302100220740ustar00rootroot00000000000000average loss = 0.3627 h vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/4a/out000066400000000000000000000000001323713302100221100ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/4b/000077500000000000000000000000001323713302100213715ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/4b/err000066400000000000000000000000271323713302100221030ustar00rootroot00000000000000average loss = 0.38117 vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/4b/out000066400000000000000000000000001323713302100221110ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/5a/000077500000000000000000000000001323713302100213715ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/5a/err000066400000000000000000000000301323713302100220750ustar00rootroot00000000000000average loss = 0.319224 vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/5a/out000066400000000000000000000000001323713302100221110ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/5b/000077500000000000000000000000001323713302100213725ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/5b/err000066400000000000000000000000301323713302100220760ustar00rootroot00000000000000average loss = 0.343701 vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/5b/out000066400000000000000000000000001323713302100221120ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/6a/000077500000000000000000000000001323713302100213725ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/6a/err000066400000000000000000000000301323713302100220760ustar00rootroot00000000000000average loss = 0.185694 vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/6a/out000066400000000000000000000000001323713302100221120ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/6b/000077500000000000000000000000001323713302100213735ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/6b/err000066400000000000000000000000301323713302100220770ustar00rootroot00000000000000average loss = 0.449856 vowpal-wabbit-8.5.0.dfsg1/big_tests/expected/6b/out000066400000000000000000000000001323713302100221130ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/testCode/000077500000000000000000000000001323713302100210355ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/big_tests/testCode/URLRep.munge.sh000077500000000000000000000001141323713302100236130ustar00rootroot00000000000000#! /bin/bash tar xzf $1 cat url_svmlight/*.svm \ | $testCodeDir/svml2vw.pl vowpal-wabbit-8.5.0.dfsg1/big_tests/testCode/covtype.munge.sh000077500000000000000000000001121323713302100241710ustar00rootroot00000000000000#! /bin/bash gunzip -c $1 \ | perl -pe 's/(.*),(.*)/$2 | $1/; s/,/ /g;' vowpal-wabbit-8.5.0.dfsg1/big_tests/testCode/floatingTolerance.pl000077500000000000000000000040211323713302100250320ustar00rootroot00000000000000#!/usr/bin/env perl #################################################################################################### # Author: I. Dan Melamed # Purpose: for each pair of floating point numbers (x,y), +ive relative tolerance r, and +ive absolute tolerance a, fail if # (x > a OR y > a) AND (x / y - 1 > r) AND (x - y > a) # Streams: 2 files of numbers; the numbers that we want to be smaller should come first # N.B.: tolerance is evaluated in only one direction, unless one of the numbers is zero # N.B.2: return code is > 0 iff tolerance test fails #################################################################################################### #check for correct usage if ($#ARGV < 0) { print "usage: $0 []\n"; exit; }; $exitCode = 0; $rtolerance = shift; $atolerance = shift; open(F, $ARGV[0]) || die "\nCouldn't open $ARGV[0]: $!\n"; shift; open(G, $ARGV[0]) || die "\nCouldn't open $ARGV[0]: $!\n"; shift; LINE: while () { @ftok = split; if (eof(G)) { print "1st file has more lines than 2nd.\n"; $exitCode = 11; last; }; $_ = ; @gtok = split; while (@ftok) { if (! @gtok) { print "Different number of tokens on line $.\n"; $exitCode = 13; next LINE; }; $ftok = shift @ftok; $gtok = shift @gtok; if ($ftok <= $atolerance && $gtok <= $atolerance) { next; }; if ( ($ftok != 0 && $gtok == 0) || ($ftok == 0 && $gtok != 0) ) { print "Difference in zeros on line $.: $ftok vs. $gtok .\n"; $exitCode = 15; next LINE; }; $rdiff = $ftok / $gtok - 1.0; $adiff = $ftok - $gtok; if ($rdiff > $rtolerance && $adiff > $atolerance) { print "Differences exceed rel. tolerance of $rtolerance and abs. tolerance of $atolerance on line $.: $ftok vs. $gtok .\n"; $exitCode = 17; next LINE; }; }; if (@gtok) { print "Different number of tokens on line $.\n"; $exitCode = 19; next LINE; }; }; if (not eof(G)) { $exitCode = 21; print "2nd file has more lines than 1st.\n"; }; exit $exitCode; vowpal-wabbit-8.5.0.dfsg1/big_tests/testCode/mnist.extract-labels.pl000077500000000000000000000010571323713302100254430ustar00rootroot00000000000000#! /usr/bin/env perl use warnings; use strict; use IO::File; local $SIG{__WARN__} = sub {}; #my $fname = shift @ARGV or die; #my $fh = new IO::File $fname, "r" or die "$fname: $!"; my $fh = \*STDIN; binmode ($fh, ':raw'); $/ = \4; my $magic = unpack ("N", <$fh>); die "wtf $magic" unless $magic == 2049; my $n_items = unpack ("N", <$fh>); warn "n_items = $n_items"; $/ = \1; while (defined ($_ = <$fh>)) { die "wtf @{[length($_)]}" unless length ($_) == 1; my ($label) = unpack ("C", $_); ++$label; print "$label 1 $label|\n"; } vowpal-wabbit-8.5.0.dfsg1/big_tests/testCode/mnist.extractfeatures.cpp000066400000000000000000000021601323713302100261020ustar00rootroot00000000000000#include #include #include #include #include #include int main (void) { using std::cin; using std::cout; using std::endl; using std::setprecision; uint32_t magic; cin.read (reinterpret_cast (&magic), sizeof (uint32_t)); magic = ntohl (magic); assert (magic == 2051); uint32_t n_images; cin.read (reinterpret_cast (&n_images), sizeof (uint32_t)); n_images = ntohl (n_images); uint32_t n_rows; cin.read (reinterpret_cast (&n_rows), sizeof (uint32_t)); n_rows = ntohl (n_rows); uint32_t n_columns; cin.read (reinterpret_cast (&n_columns), sizeof (uint32_t)); n_columns = ntohl (n_columns); uint32_t rc = n_rows * n_columns; unsigned char buf[rc]; for (cin.read (reinterpret_cast (buf), rc); ! cin.eof (); cin.read (reinterpret_cast (buf), rc)) { cout << "|p"; for (unsigned int p = 0; p < n_rows * n_columns; ++p) { if (buf[p]) cout << " " << p << ":" << setprecision (8) << static_cast(buf[p])/256.0; } cout << endl; } return 0; } vowpal-wabbit-8.5.0.dfsg1/big_tests/testCode/mnist.extractpixels.cpp000066400000000000000000000021111323713302100255640ustar00rootroot00000000000000#include #include #include #include #include #include int main (void) { using std::cin; using std::cout; using std::endl; using std::setprecision; uint32_t magic; cin.read (reinterpret_cast (&magic), sizeof (uint32_t)); magic = ntohl (magic); assert (magic == 2051); uint32_t n_images; cin.read (reinterpret_cast (&n_images), sizeof (uint32_t)); n_images = ntohl (n_images); uint32_t n_rows; cin.read (reinterpret_cast (&n_rows), sizeof (uint32_t)); n_rows = ntohl (n_rows); uint32_t n_columns; cin.read (reinterpret_cast (&n_columns), sizeof (uint32_t)); n_columns = ntohl (n_columns); uint32_t rc = n_rows * n_columns; unsigned char buf[rc]; for (cin.read (reinterpret_cast (buf), rc); ! cin.eof (); cin.read (reinterpret_cast (buf), rc)) { for (unsigned int p = 0; p < n_rows * n_columns; ++p) { if (buf[p]) cout << " " << p << ":" << static_cast(buf[p]); } cout << endl; } return 0; } vowpal-wabbit-8.5.0.dfsg1/big_tests/testCode/mnist.munge.sh000077500000000000000000000002141323713302100236350ustar00rootroot00000000000000#! /bin/bash paste -d' ' \ <(gunzip -c $1 | $testCodeDir/mnist.extract-labels.pl) \ <(gunzip -c $2 | $testCodeDir/mnist.extractfeatures) vowpal-wabbit-8.5.0.dfsg1/big_tests/testCode/movielensRatings2vw.pl000077500000000000000000000024171323713302100253710ustar00rootroot00000000000000#! /usr/bin/env perl use IO::File; use warnings; use strict; srand 69; sub output_user ($$@) { my ($trainfh, $testfh, @rows) = @_; return () unless @rows > 1; my @permrows = map { $_->[1] } sort { $a->[0] <=> $b->[0] } map { [ rand (), $_ ] } @rows; my @testrows = splice @permrows, -1; print $trainfh @permrows; return @testrows; } my $trainfile = shift @ARGV or die; my $testfile = shift @ARGV or die; my $trainfh = new IO::File $trainfile, "w" or die; my $testfh = new IO::File $testfile, "w" or die; my $olduser; my @rows; my @save; my %seen; while (defined ($_ = <>)) { chomp; my ($user, $movie, $rating, undef) = split /::/, $_; if (defined ($olduser) && $user != $olduser) { push @save, output_user ($trainfh, $testfh, @rows); undef @rows; die "input file not collated" if $seen{$olduser}++; } push @rows, "$rating $rating|user $user |movie $movie\n"; $olduser = $user; } push @save, output_user ($trainfh, $testfh, @rows); my @permsave = map { $_->[1] } sort { $a->[0] <=> $b->[0] } map { [ rand (), $_ ] } @save; my @test = splice @permsave, 0, 5000; print $trainfh @permsave; print $testfh @test; vowpal-wabbit-8.5.0.dfsg1/big_tests/testCode/ocr2vw.py000077500000000000000000000043021323713302100226330ustar00rootroot00000000000000#!/usr/bin/env python # convert letter.data to letter.vw def read_letter_names (fn): ret = list() with open(fn) as ins: for line in ins: ret.append(line.rstrip()) print "Read %d names from %s" % (len(ret),fn) return ret def find_pixel_start (names): for i in range(len(names)): if names[i].startswith("p_"): return i raise ValueError("No pixel data",names) def data2vw (ifn, train, test, names): lineno = 0 trainN = 0 testN = 0 if ifn.endswith(".gz"): import gzip iopener = gzip.open else: iopener = open id_pos = names.index("id") letter_pos = names.index("letter") pixel_start = find_pixel_start(names) with iopener(ifn) as ins, open(train,"wb") as trainS, open(test,"wb") as testS: for line in ins: lineno += 1 vals = line.rstrip().split('\t') if len(vals) != len(names): raise ValueError("Bad field count", len(vals),len(names),vals,names) char = vals[letter_pos] if len(char) != 1: raise ValueError("Bad letter",char) if lineno % 10 == 0: testN += 1 outs = testS else: trainN += 1 outs = trainS outs.write("%d 1 %s-%s|Pixel" % (ord(char)-ord('a')+1,char,vals[id_pos])) for i in range(pixel_start,len(names)): if vals[i] != '0': outs.write(' %s:%s' % (names[i],vals[i])) outs.write('\n') print "Read %d lines from %s; wrote %d lines into %s and %d lines into %s" % ( lineno,ifn,trainN,train,testN,test) if __name__ == '__main__': import argparse parser = argparse.ArgumentParser(description='Convert letters.data to VW format') parser.add_argument('input',help='path to letter.data[.gz]') parser.add_argument('names',help='path to letter.names') parser.add_argument('train',help='VW train file location (90%)') parser.add_argument('test',help='VW test file location (10%)') args = parser.parse_args() data2vw(args.input,args.train,args.test,read_letter_names(args.names)) vowpal-wabbit-8.5.0.dfsg1/big_tests/testCode/shuffle.pl000077500000000000000000000006501323713302100230320ustar00rootroot00000000000000#!/usr/bin/env perl ######################################################################## # shuffle input lines # same as random sample (without replacement) of all of them ######################################################################## srand(13); while (<>) { $line[$. - 1] = $_; }; $size = $count = $.; for(;$size > 0; $size-- && $count--) { $ind = int rand $count; print splice(@line, $ind, 1); }; vowpal-wabbit-8.5.0.dfsg1/big_tests/testCode/svml2vw.pl000077500000000000000000000002071323713302100230140ustar00rootroot00000000000000#!/usr/bin/env perl # convert data format from SVM-Light to VW while (<>) { s/^\-1/0/; s/^\+1/1/; s/ / | /; print; } vowpal-wabbit-8.5.0.dfsg1/big_tests/testDetails.mk000066400000000000000000000036131323713302100221040ustar00rootroot00000000000000 # MNIST training 1a.inData := $(dataDir)/mnist.dir/train.prep 1a.params := --oaa 10 -d $(1a.inData) -f $(stageDir)/1a.dir/mnist.model -b 24 --adaptive --invariant --holdout_off -l 0.1 --nn 40 --passes 24 -k --compressed --cache_file $(stageDir)/1a.dir/mnist.cache # MNIST prediction 1b.inData := $(dataDir)/mnist.dir/test.prep 1b.params := -t -d $(1b.inData) -i $(stageDir)/1a.dir/mnist.model # test dependencies not working yet 1b.deps := 1a.valid # COVERTYPE 2.inData := $(dataDir)/covtype.dir/prep 2.params := --oaa 7 -d $(2.inData) # URL Reputation 3.inData := $(dataDir)/URLRep.dir/prep 3.params := -d $(3.inData) # Entity Relation training 4a.inData := $(dataDir)/ER.dir/train.prep 4a.params := -b 24 -d $(4a.inData) --search 10 --passes 10 --search_task entity_relation --constraints --search_alpha 1e-8 -f $(stageDir)/4a.dir/er.model --cache_file $(stageDir)/4a.dir/er.cache # Entity Relation prediction 4b.inData := $(dataDir)/ER.dir/test.prep 4b.params := -t -d $(4b.inData) -i $(stageDir)/4a.dir/er.model 4b.deps := 4a.valid # MovieLens training 5a.inData := $(dataDir)/movielens.dir/train.prep 5a.params := --loss_function quantile -l 0.45 -b 24 --passes 100 -k --cache_file $(stageDir)/5a.dir/movielens.cache -d $(5a.inData) --holdout_off --lrq um14 --lrqdropout --adaptive --invariant -f $(stageDir)/5a.dir/movielens.model # MovieLens prediction 5b.inData := $(dataDir)/movielens.dir/test.prep 5b.params := --loss_function quantile -t -i $(stageDir)/5a.dir/movielens.model -d $(5b.inData) 5b.deps := 5a.valid # OCR training 6a.inData := $(dataDir)/OCR.dir/train.prep 6a.params := -d $(6a.inData) -f $(stageDir)/6a.dir/OCR.model --cache_file $(stageDir)/6a.dir/OCR.cache -k --oaa 26 --adaptive --invariant --holdout_off --loss_function logistic --passes 14 # OCR prediction 6b.inData := $(dataDir)/OCR.dir/test.prep 6b.params := -i $(stageDir)/6a.dir/OCR.model -d $(6b.inData) --testonly 6b.deps := 6a.valid vowpal-wabbit-8.5.0.dfsg1/big_tests/testGroups.mk000066400000000000000000000002061323713302100217710ustar00rootroot00000000000000# this test group is bogus, for illustration only; adjust if you know what you're doing regression_group: 1a.valid 1b.valid 3.valid ; vowpal-wabbit-8.5.0.dfsg1/c_test/000077500000000000000000000000001323713302100165615ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/c_test/c_test.vcxproj000066400000000000000000000310141323713302100214560ustar00rootroot00000000000000 DebugLeakCheck Win32 DebugLeakCheck x64 Debug Win32 Debug x64 Release Win32 Release x64 {E5865596-E5F0-4CA3-B04A-4E34B798744A} c_test ..\vowpalwabbit\ ..\sdl\SDL-7.0-Recommended.ruleset true Application true MultiByte v140 Application true MultiByte v140 Application true MultiByte v140 Application true MultiByte v140 Application false true MultiByte v140 Application false true MultiByte v140 $(SolutionDir)dll\$(Platform)\$(ConfigurationName);$(LibraryPath) $(ProjectDir)Bin\$(Platform)\$(Configuration)\ $(SolutionDir)dll\$(Platform)\$(ConfigurationName);$(LibraryPath) $(ProjectDir)Bin\$(Platform)\$(Configuration)\ $(ProjectDir)Bin\$(Platform)\$(Configuration)\ $(ProjectDir)Bin\$(Platform)\$(Configuration)\ $(ProjectDir)Bin\$(Platform)\$(Configuration)\ $(ProjectDir)Bin\$(Platform)\$(Configuration)\ Level3 Disabled $(SolutionDir);%(AdditionalIncludeDirectories) true $(SolutionDir)dll\$(PlatformShortName)\$(ConfigurationName)\libvw.lib;%(AdditionalDependencies) xcopy /y/d $(SolutionDir)dll\$(PlatformShortName)\$(ConfigurationName)\libvw.* $(TargetDir) Level3 Disabled $(SolutionDir);%(AdditionalIncludeDirectories) true $(SolutionDir)dll\$(PlatformShortName)\$(ConfigurationName)\libvw.lib;%(AdditionalDependencies) xcopy /y/d $(SolutionDir)dll\$(PlatformShortName)\$(ConfigurationName)\libvw.* $(TargetDir) Level3 Disabled $(SolutionDir);%(AdditionalIncludeDirectories) true $(SolutionDir)dll\$(Platform)\$(ConfigurationName)\libvw.lib;%(AdditionalDependencies) xcopy /y/d $(SolutionDir)dll\$(PlatformShortName)\$(ConfigurationName)\libvw.* $(TargetDir) Level3 Disabled $(SolutionDir);%(AdditionalIncludeDirectories) true $(SolutionDir)dll\$(Platform)\$(ConfigurationName)\libvw.lib;%(AdditionalDependencies) xcopy /y/d $(SolutionDir)dll\$(PlatformShortName)\$(ConfigurationName)\libvw.* $(TargetDir) Level3 MaxSpeed true true $(SolutionDir);%(AdditionalIncludeDirectories) true true true $(SolutionDir)dll\$(PlatformShortName)\$(ConfigurationName)\libvw.lib;%(AdditionalDependencies) xcopy /y/d $(SolutionDir)dll\$(PlatformShortName)\$(ConfigurationName)\libvw.* $(TargetDir) Level3 MaxSpeed true true $(SolutionDir);%(AdditionalIncludeDirectories) true true true $(SolutionDir)dll\$(Platform)\$(Configuration)\libvw.lib;%(AdditionalDependencies) xcopy /y/d $(SolutionDir)dll\$(PlatformName)\$(ConfigurationName)\libvw.* $(TargetDir) This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. vowpal-wabbit-8.5.0.dfsg1/c_test/sample.c000066400000000000000000000015471323713302100202150ustar00rootroot00000000000000#include #ifndef OMIT_MS // a gcc sample would omit these two defines #define MS_CONV // use Microsoft libraray convention #define USE_CODECVT // enable wide character converesion enabled API #endif /* To compile with gcc use this command LD_LIBRARY_PATH=../vowpalwabbit/.libs ; gcc sample_gcc.c -I./../vowpalwabbit/ -L./../vowpalwabbit/.libs -lvw -lvw_c_wrapper -lallreduce -o sample_gcc */ typedef short char16_t; #define bool int #define true (1) #define false (0) #include "vwdll.h" int main() { VW_HANDLE vw; VW_EXAMPLE example; float score; printf("this is a native c program calling vw\n"); vw = VW_InitializeA("-q st --noconstant --quiet"); example = VW_ReadExampleA(vw, "1 |s p^the_man w^the w^man |t p^un_homme w^un w^homme"); score = VW_Learn(vw, example); VW_Finish(vw); printf("Score = %f\n", score); return 0; }vowpal-wabbit-8.5.0.dfsg1/cluster/000077500000000000000000000000001323713302100167615ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cluster/Makefile.am000066400000000000000000000011001323713302100210050ustar00rootroot00000000000000bin_PROGRAMS = spanning_tree spanning_tree_SOURCES = spanning_tree_main.cc ../vowpalwabbit/spanning_tree.cc ../vowpalwabbit/vw_exception.cc spanning_tree_LDADD = ../vowpalwabbit/liballreduce.la spanning_tree_DEPENDENCIES = ../vowpalwabbit/liballreduce.la ACLOCAL_AMFLAGS = -I acinclude.d AM_CPPFLAGS = -I$(top_srcdir)/vowpalwabbit AM_CXXFLAGS = ${BOOST_CPPFLAGS} ${ZLIB_CPPFLAGS} ${PTHREAD_CFLAGS} AM_LDFLAGS = ${BOOST_LDFLAGS} ${BOOST_PROGRAM_OPTIONS_LIB} ${ZLIB_LDFLAGS} ${PTHREAD_LIBS} if CLANG_LIBCXX AM_CXXFLAGS += -stdlib=libc++ -I ../vowpalwabbit -std=c++0x endif vowpal-wabbit-8.5.0.dfsg1/cluster/Makefile.in000066400000000000000000000525171323713302100210400ustar00rootroot00000000000000# Makefile.in generated by automake 1.15 from Makefile.am. # @configure_input@ # Copyright (C) 1994-2014 Free Software Foundation, Inc. # This Makefile.in is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY, to the extent permitted by law; without # even the implied warranty of MERCHANTABILITY or FITNESS FOR A # PARTICULAR PURPOSE. @SET_MAKE@ VPATH = @srcdir@ am__is_gnu_make = { \ if test -z '$(MAKELEVEL)'; then \ false; \ elif test -n '$(MAKE_HOST)'; then \ true; \ elif test -n '$(MAKE_VERSION)' && test -n '$(CURDIR)'; then \ true; \ else \ false; \ fi; \ } am__make_running_with_option = \ case $${target_option-} in \ ?) ;; \ *) echo "am__make_running_with_option: internal error: invalid" \ "target option '$${target_option-}' specified" >&2; \ exit 1;; \ esac; \ has_opt=no; \ sane_makeflags=$$MAKEFLAGS; \ if $(am__is_gnu_make); then \ sane_makeflags=$$MFLAGS; \ else \ case $$MAKEFLAGS in \ *\\[\ \ ]*) \ bs=\\; \ sane_makeflags=`printf '%s\n' "$$MAKEFLAGS" \ | sed "s/$$bs$$bs[$$bs $$bs ]*//g"`;; \ esac; \ fi; \ skip_next=no; \ strip_trailopt () \ { \ flg=`printf '%s\n' "$$flg" | sed "s/$$1.*$$//"`; \ }; \ for flg in $$sane_makeflags; do \ test $$skip_next = yes && { skip_next=no; continue; }; \ case $$flg in \ *=*|--*) continue;; \ -*I) strip_trailopt 'I'; skip_next=yes;; \ -*I?*) strip_trailopt 'I';; \ -*O) strip_trailopt 'O'; skip_next=yes;; \ -*O?*) strip_trailopt 'O';; \ -*l) strip_trailopt 'l'; skip_next=yes;; \ -*l?*) strip_trailopt 'l';; \ -[dEDm]) skip_next=yes;; \ -[JT]) skip_next=yes;; \ esac; \ case $$flg in \ *$$target_option*) has_opt=yes; break;; \ esac; \ done; \ test $$has_opt = yes am__make_dryrun = (target_option=n; $(am__make_running_with_option)) am__make_keepgoing = (target_option=k; $(am__make_running_with_option)) pkgdatadir = $(datadir)/@PACKAGE@ pkgincludedir = $(includedir)/@PACKAGE@ pkglibdir = $(libdir)/@PACKAGE@ pkglibexecdir = $(libexecdir)/@PACKAGE@ am__cd = CDPATH="$${ZSH_VERSION+.}$(PATH_SEPARATOR)" && cd install_sh_DATA = $(install_sh) -c -m 644 install_sh_PROGRAM = $(install_sh) -c install_sh_SCRIPT = $(install_sh) -c INSTALL_HEADER = $(INSTALL_DATA) transform = $(program_transform_name) NORMAL_INSTALL = : PRE_INSTALL = : POST_INSTALL = : NORMAL_UNINSTALL = : PRE_UNINSTALL = : POST_UNINSTALL = : build_triplet = @build@ host_triplet = @host@ bin_PROGRAMS = spanning_tree$(EXEEXT) @CLANG_LIBCXX_TRUE@am__append_1 = -stdlib=libc++ -I ../vowpalwabbit -std=c++0x subdir = cluster ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 am__aclocal_m4_deps = $(top_srcdir)/acinclude.d/ax_boost_base.m4 \ $(top_srcdir)/acinclude.d/ax_boost_program_options.m4 \ $(top_srcdir)/acinclude.d/ax_check_zlib.m4 \ $(top_srcdir)/acinclude.d/ax_cxx_compile_stdcxx_11.m4 \ $(top_srcdir)/acinclude.d/ax_pthread.m4 \ $(top_srcdir)/acinclude.d/libtool.m4 \ $(top_srcdir)/acinclude.d/ltoptions.m4 \ $(top_srcdir)/acinclude.d/ltsugar.m4 \ $(top_srcdir)/acinclude.d/ltversion.m4 \ $(top_srcdir)/acinclude.d/lt~obsolete.m4 \ $(top_srcdir)/configure.ac am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \ $(ACLOCAL_M4) DIST_COMMON = $(srcdir)/Makefile.am $(am__DIST_COMMON) mkinstalldirs = $(install_sh) -d CONFIG_HEADER = $(top_builddir)/vowpalwabbit/config.h CONFIG_CLEAN_FILES = CONFIG_CLEAN_VPATH_FILES = am__installdirs = "$(DESTDIR)$(bindir)" PROGRAMS = $(bin_PROGRAMS) am__dirstamp = $(am__leading_dot)dirstamp am_spanning_tree_OBJECTS = spanning_tree_main.$(OBJEXT) \ ../vowpalwabbit/spanning_tree.$(OBJEXT) \ ../vowpalwabbit/vw_exception.$(OBJEXT) spanning_tree_OBJECTS = $(am_spanning_tree_OBJECTS) AM_V_lt = $(am__v_lt_@AM_V@) am__v_lt_ = $(am__v_lt_@AM_DEFAULT_V@) am__v_lt_0 = --silent am__v_lt_1 = AM_V_P = $(am__v_P_@AM_V@) am__v_P_ = $(am__v_P_@AM_DEFAULT_V@) am__v_P_0 = false am__v_P_1 = : AM_V_GEN = $(am__v_GEN_@AM_V@) am__v_GEN_ = $(am__v_GEN_@AM_DEFAULT_V@) am__v_GEN_0 = @echo " GEN " $@; am__v_GEN_1 = AM_V_at = $(am__v_at_@AM_V@) am__v_at_ = $(am__v_at_@AM_DEFAULT_V@) am__v_at_0 = @ am__v_at_1 = DEFAULT_INCLUDES = -I.@am__isrc@ -I$(top_builddir)/vowpalwabbit depcomp = $(SHELL) $(top_srcdir)/depcomp am__depfiles_maybe = depfiles am__mv = mv -f CXXCOMPILE = $(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) \ $(AM_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) LTCXXCOMPILE = $(LIBTOOL) $(AM_V_lt) --tag=CXX $(AM_LIBTOOLFLAGS) \ $(LIBTOOLFLAGS) --mode=compile $(CXX) $(DEFS) \ $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) \ $(AM_CXXFLAGS) $(CXXFLAGS) AM_V_CXX = $(am__v_CXX_@AM_V@) am__v_CXX_ = $(am__v_CXX_@AM_DEFAULT_V@) am__v_CXX_0 = @echo " CXX " $@; am__v_CXX_1 = CXXLD = $(CXX) CXXLINK = $(LIBTOOL) $(AM_V_lt) --tag=CXX $(AM_LIBTOOLFLAGS) \ $(LIBTOOLFLAGS) --mode=link $(CXXLD) $(AM_CXXFLAGS) \ $(CXXFLAGS) $(AM_LDFLAGS) $(LDFLAGS) -o $@ AM_V_CXXLD = $(am__v_CXXLD_@AM_V@) am__v_CXXLD_ = $(am__v_CXXLD_@AM_DEFAULT_V@) am__v_CXXLD_0 = @echo " CXXLD " $@; am__v_CXXLD_1 = SOURCES = $(spanning_tree_SOURCES) DIST_SOURCES = $(spanning_tree_SOURCES) am__can_run_installinfo = \ case $$AM_UPDATE_INFO_DIR in \ n|no|NO) false;; \ *) (install-info --version) >/dev/null 2>&1;; \ esac am__tagged_files = $(HEADERS) $(SOURCES) $(TAGS_FILES) $(LISP) # Read a list of newline-separated strings from the standard input, # and print each of them once, without duplicates. Input order is # *not* preserved. am__uniquify_input = $(AWK) '\ BEGIN { nonempty = 0; } \ { items[$$0] = 1; nonempty = 1; } \ END { if (nonempty) { for (i in items) print i; }; } \ ' # Make sure the list of sources is unique. This is necessary because, # e.g., the same source file might be shared among _SOURCES variables # for different programs/libraries. am__define_uniq_tagged_files = \ list='$(am__tagged_files)'; \ unique=`for i in $$list; do \ if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ done | $(am__uniquify_input)` ETAGS = etags CTAGS = ctags am__DIST_COMMON = $(srcdir)/Makefile.in $(top_srcdir)/depcomp DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST) ACLOCAL = @ACLOCAL@ AMTAR = @AMTAR@ AM_DEFAULT_VERBOSITY = @AM_DEFAULT_VERBOSITY@ AR = @AR@ AUTOCONF = @AUTOCONF@ AUTOHEADER = @AUTOHEADER@ AUTOMAKE = @AUTOMAKE@ AWK = @AWK@ BOOST_CPPFLAGS = @BOOST_CPPFLAGS@ BOOST_LDFLAGS = @BOOST_LDFLAGS@ BOOST_PROGRAM_OPTIONS_LIB = @BOOST_PROGRAM_OPTIONS_LIB@ CC = @CC@ CCDEPMODE = @CCDEPMODE@ CFLAGS = @CFLAGS@ CPP = @CPP@ CPPFLAGS = @CPPFLAGS@ CXX = @CXX@ CXXCPP = @CXXCPP@ CXXDEPMODE = @CXXDEPMODE@ CXXFLAGS = @CXXFLAGS@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DLLTOOL = @DLLTOOL@ DSYMUTIL = @DSYMUTIL@ DUMPBIN = @DUMPBIN@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ ECHO_T = @ECHO_T@ EGREP = @EGREP@ EXEEXT = @EXEEXT@ FGREP = @FGREP@ GREP = @GREP@ HAVE_CXX11 = @HAVE_CXX11@ INSTALL = @INSTALL@ INSTALL_DATA = @INSTALL_DATA@ INSTALL_PROGRAM = @INSTALL_PROGRAM@ INSTALL_SCRIPT = @INSTALL_SCRIPT@ INSTALL_STRIP_PROGRAM = @INSTALL_STRIP_PROGRAM@ LD = @LD@ LDFLAGS = @LDFLAGS@ LIBOBJS = @LIBOBJS@ LIBS = @LIBS@ LIBTOOL = @LIBTOOL@ LIPO = @LIPO@ LN_S = @LN_S@ LTLIBOBJS = @LTLIBOBJS@ LT_SYS_LIBRARY_PATH = @LT_SYS_LIBRARY_PATH@ MAKEINFO = @MAKEINFO@ MANIFEST_TOOL = @MANIFEST_TOOL@ MKDIR_P = @MKDIR_P@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ OTOOL = @OTOOL@ OTOOL64 = @OTOOL64@ PACKAGE = @PACKAGE@ PACKAGE_BUGREPORT = @PACKAGE_BUGREPORT@ PACKAGE_NAME = @PACKAGE_NAME@ PACKAGE_STRING = @PACKAGE_STRING@ PACKAGE_TARNAME = @PACKAGE_TARNAME@ PACKAGE_URL = @PACKAGE_URL@ PACKAGE_VERSION = @PACKAGE_VERSION@ PATH_SEPARATOR = @PATH_SEPARATOR@ PTHREAD_CC = @PTHREAD_CC@ PTHREAD_CFLAGS = @PTHREAD_CFLAGS@ PTHREAD_LIBS = @PTHREAD_LIBS@ RANLIB = @RANLIB@ SED = @SED@ SET_MAKE = @SET_MAKE@ SHELL = @SHELL@ STRIP = @STRIP@ VERSION = @VERSION@ ZLIB_CPPFLAGS = @ZLIB_CPPFLAGS@ ZLIB_LDFLAGS = @ZLIB_LDFLAGS@ abs_builddir = @abs_builddir@ abs_srcdir = @abs_srcdir@ abs_top_builddir = @abs_top_builddir@ abs_top_srcdir = @abs_top_srcdir@ ac_ct_AR = @ac_ct_AR@ ac_ct_CC = @ac_ct_CC@ ac_ct_CXX = @ac_ct_CXX@ ac_ct_DUMPBIN = @ac_ct_DUMPBIN@ am__include = @am__include@ am__leading_dot = @am__leading_dot@ am__quote = @am__quote@ am__tar = @am__tar@ am__untar = @am__untar@ ax_pthread_config = @ax_pthread_config@ bindir = @bindir@ build = @build@ build_alias = @build_alias@ build_cpu = @build_cpu@ build_os = @build_os@ build_vendor = @build_vendor@ builddir = @builddir@ datadir = @datadir@ datarootdir = @datarootdir@ docdir = @docdir@ dvidir = @dvidir@ exec_prefix = @exec_prefix@ host = @host@ host_alias = @host_alias@ host_cpu = @host_cpu@ host_os = @host_os@ host_vendor = @host_vendor@ htmldir = @htmldir@ includedir = @includedir@ infodir = @infodir@ install_sh = @install_sh@ libdir = @libdir@ libexecdir = @libexecdir@ localedir = @localedir@ localstatedir = @localstatedir@ mandir = @mandir@ mkdir_p = @mkdir_p@ oldincludedir = @oldincludedir@ pdfdir = @pdfdir@ prefix = @prefix@ program_transform_name = @program_transform_name@ psdir = @psdir@ runstatedir = @runstatedir@ sbindir = @sbindir@ sharedstatedir = @sharedstatedir@ srcdir = @srcdir@ sysconfdir = @sysconfdir@ target_alias = @target_alias@ top_build_prefix = @top_build_prefix@ top_builddir = @top_builddir@ top_srcdir = @top_srcdir@ spanning_tree_SOURCES = spanning_tree_main.cc ../vowpalwabbit/spanning_tree.cc ../vowpalwabbit/vw_exception.cc spanning_tree_LDADD = ../vowpalwabbit/liballreduce.la spanning_tree_DEPENDENCIES = ../vowpalwabbit/liballreduce.la ACLOCAL_AMFLAGS = -I acinclude.d AM_CPPFLAGS = -I$(top_srcdir)/vowpalwabbit AM_CXXFLAGS = ${BOOST_CPPFLAGS} ${ZLIB_CPPFLAGS} ${PTHREAD_CFLAGS} \ $(am__append_1) AM_LDFLAGS = ${BOOST_LDFLAGS} ${BOOST_PROGRAM_OPTIONS_LIB} ${ZLIB_LDFLAGS} ${PTHREAD_LIBS} all: all-am .SUFFIXES: .SUFFIXES: .cc .lo .o .obj $(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps) @for dep in $?; do \ case '$(am__configure_deps)' in \ *$$dep*) \ ( cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh ) \ && { if test -f $@; then exit 0; else break; fi; }; \ exit 1;; \ esac; \ done; \ echo ' cd $(top_srcdir) && $(AUTOMAKE) --foreign cluster/Makefile'; \ $(am__cd) $(top_srcdir) && \ $(AUTOMAKE) --foreign cluster/Makefile Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status @case '$?' in \ *config.status*) \ cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh;; \ *) \ echo ' cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@ $(am__depfiles_maybe)'; \ cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@ $(am__depfiles_maybe);; \ esac; $(top_builddir)/config.status: $(top_srcdir)/configure $(CONFIG_STATUS_DEPENDENCIES) cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh $(top_srcdir)/configure: $(am__configure_deps) cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh $(ACLOCAL_M4): $(am__aclocal_m4_deps) cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh $(am__aclocal_m4_deps): install-binPROGRAMS: $(bin_PROGRAMS) @$(NORMAL_INSTALL) @list='$(bin_PROGRAMS)'; test -n "$(bindir)" || list=; \ if test -n "$$list"; then \ echo " $(MKDIR_P) '$(DESTDIR)$(bindir)'"; \ $(MKDIR_P) "$(DESTDIR)$(bindir)" || exit 1; \ fi; \ for p in $$list; do echo "$$p $$p"; done | \ sed 's/$(EXEEXT)$$//' | \ while read p p1; do if test -f $$p \ || test -f $$p1 \ ; then echo "$$p"; echo "$$p"; else :; fi; \ done | \ sed -e 'p;s,.*/,,;n;h' \ -e 's|.*|.|' \ -e 'p;x;s,.*/,,;s/$(EXEEXT)$$//;$(transform);s/$$/$(EXEEXT)/' | \ sed 'N;N;N;s,\n, ,g' | \ $(AWK) 'BEGIN { files["."] = ""; dirs["."] = 1 } \ { d=$$3; if (dirs[d] != 1) { print "d", d; dirs[d] = 1 } \ if ($$2 == $$4) files[d] = files[d] " " $$1; \ else { print "f", $$3 "/" $$4, $$1; } } \ END { for (d in files) print "f", d, files[d] }' | \ while read type dir files; do \ if test "$$dir" = .; then dir=; else dir=/$$dir; fi; \ test -z "$$files" || { \ echo " $(INSTALL_PROGRAM_ENV) $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL_PROGRAM) $$files '$(DESTDIR)$(bindir)$$dir'"; \ $(INSTALL_PROGRAM_ENV) $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL_PROGRAM) $$files "$(DESTDIR)$(bindir)$$dir" || exit $$?; \ } \ ; done uninstall-binPROGRAMS: @$(NORMAL_UNINSTALL) @list='$(bin_PROGRAMS)'; test -n "$(bindir)" || list=; \ files=`for p in $$list; do echo "$$p"; done | \ sed -e 'h;s,^.*/,,;s/$(EXEEXT)$$//;$(transform)' \ -e 's/$$/$(EXEEXT)/' \ `; \ test -n "$$list" || exit 0; \ echo " ( cd '$(DESTDIR)$(bindir)' && rm -f" $$files ")"; \ cd "$(DESTDIR)$(bindir)" && rm -f $$files clean-binPROGRAMS: @list='$(bin_PROGRAMS)'; test -n "$$list" || exit 0; \ echo " rm -f" $$list; \ rm -f $$list || exit $$?; \ test -n "$(EXEEXT)" || exit 0; \ list=`for p in $$list; do echo "$$p"; done | sed 's/$(EXEEXT)$$//'`; \ echo " rm -f" $$list; \ rm -f $$list ../vowpalwabbit/$(am__dirstamp): @$(MKDIR_P) ../vowpalwabbit @: > ../vowpalwabbit/$(am__dirstamp) ../vowpalwabbit/$(DEPDIR)/$(am__dirstamp): @$(MKDIR_P) ../vowpalwabbit/$(DEPDIR) @: > ../vowpalwabbit/$(DEPDIR)/$(am__dirstamp) ../vowpalwabbit/spanning_tree.$(OBJEXT): \ ../vowpalwabbit/$(am__dirstamp) \ ../vowpalwabbit/$(DEPDIR)/$(am__dirstamp) ../vowpalwabbit/vw_exception.$(OBJEXT): \ ../vowpalwabbit/$(am__dirstamp) \ ../vowpalwabbit/$(DEPDIR)/$(am__dirstamp) spanning_tree$(EXEEXT): $(spanning_tree_OBJECTS) $(spanning_tree_DEPENDENCIES) $(EXTRA_spanning_tree_DEPENDENCIES) @rm -f spanning_tree$(EXEEXT) $(AM_V_CXXLD)$(CXXLINK) $(spanning_tree_OBJECTS) $(spanning_tree_LDADD) $(LIBS) mostlyclean-compile: -rm -f *.$(OBJEXT) -rm -f ../vowpalwabbit/*.$(OBJEXT) distclean-compile: -rm -f *.tab.c @AMDEP_TRUE@@am__include@ @am__quote@../vowpalwabbit/$(DEPDIR)/spanning_tree.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@../vowpalwabbit/$(DEPDIR)/vw_exception.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/spanning_tree_main.Po@am__quote@ .cc.o: @am__fastdepCXX_TRUE@ $(AM_V_CXX)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ @am__fastdepCXX_TRUE@ $(CXXCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ @am__fastdepCXX_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXXCOMPILE) -c -o $@ $< .cc.obj: @am__fastdepCXX_TRUE@ $(AM_V_CXX)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ @am__fastdepCXX_TRUE@ $(CXXCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ @am__fastdepCXX_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXXCOMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .cc.lo: @am__fastdepCXX_TRUE@ $(AM_V_CXX)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ @am__fastdepCXX_TRUE@ $(LTCXXCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ @am__fastdepCXX_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(LTCXXCOMPILE) -c -o $@ $< mostlyclean-libtool: -rm -f *.lo clean-libtool: -rm -rf .libs _libs ID: $(am__tagged_files) $(am__define_uniq_tagged_files); mkid -fID $$unique tags: tags-am TAGS: tags tags-am: $(TAGS_DEPENDENCIES) $(am__tagged_files) set x; \ here=`pwd`; \ $(am__define_uniq_tagged_files); \ shift; \ if test -z "$(ETAGS_ARGS)$$*$$unique"; then :; else \ test -n "$$unique" || unique=$$empty_fix; \ if test $$# -gt 0; then \ $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ "$$@" $$unique; \ else \ $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ $$unique; \ fi; \ fi ctags: ctags-am CTAGS: ctags ctags-am: $(TAGS_DEPENDENCIES) $(am__tagged_files) $(am__define_uniq_tagged_files); \ test -z "$(CTAGS_ARGS)$$unique" \ || $(CTAGS) $(CTAGSFLAGS) $(AM_CTAGSFLAGS) $(CTAGS_ARGS) \ $$unique GTAGS: here=`$(am__cd) $(top_builddir) && pwd` \ && $(am__cd) $(top_srcdir) \ && gtags -i $(GTAGS_ARGS) "$$here" cscopelist: cscopelist-am cscopelist-am: $(am__tagged_files) list='$(am__tagged_files)'; \ case "$(srcdir)" in \ [\\/]* | ?:[\\/]*) sdir="$(srcdir)" ;; \ *) sdir=$(subdir)/$(srcdir) ;; \ esac; \ for i in $$list; do \ if test -f "$$i"; then \ echo "$(subdir)/$$i"; \ else \ echo "$$sdir/$$i"; \ fi; \ done >> $(top_builddir)/cscope.files distclean-tags: -rm -f TAGS ID GTAGS GRTAGS GSYMS GPATH tags distdir: $(DISTFILES) @srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ topsrcdirstrip=`echo "$(top_srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ list='$(DISTFILES)'; \ dist_files=`for file in $$list; do echo $$file; done | \ sed -e "s|^$$srcdirstrip/||;t" \ -e "s|^$$topsrcdirstrip/|$(top_builddir)/|;t"`; \ case $$dist_files in \ */*) $(MKDIR_P) `echo "$$dist_files" | \ sed '/\//!d;s|^|$(distdir)/|;s,/[^/]*$$,,' | \ sort -u` ;; \ esac; \ for file in $$dist_files; do \ if test -f $$file || test -d $$file; then d=.; else d=$(srcdir); fi; \ if test -d $$d/$$file; then \ dir=`echo "/$$file" | sed -e 's,/[^/]*$$,,'`; \ if test -d "$(distdir)/$$file"; then \ find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ fi; \ if test -d $(srcdir)/$$file && test $$d != $(srcdir); then \ cp -fpR $(srcdir)/$$file "$(distdir)$$dir" || exit 1; \ find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ fi; \ cp -fpR $$d/$$file "$(distdir)$$dir" || exit 1; \ else \ test -f "$(distdir)/$$file" \ || cp -p $$d/$$file "$(distdir)/$$file" \ || exit 1; \ fi; \ done check-am: all-am check: check-am all-am: Makefile $(PROGRAMS) installdirs: for dir in "$(DESTDIR)$(bindir)"; do \ test -z "$$dir" || $(MKDIR_P) "$$dir"; \ done install: install-am install-exec: install-exec-am install-data: install-data-am uninstall: uninstall-am install-am: all-am @$(MAKE) $(AM_MAKEFLAGS) install-exec-am install-data-am installcheck: installcheck-am install-strip: if test -z '$(STRIP)'; then \ $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ install; \ else \ $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ "INSTALL_PROGRAM_ENV=STRIPPROG='$(STRIP)'" install; \ fi mostlyclean-generic: clean-generic: distclean-generic: -test -z "$(CONFIG_CLEAN_FILES)" || rm -f $(CONFIG_CLEAN_FILES) -test . = "$(srcdir)" || test -z "$(CONFIG_CLEAN_VPATH_FILES)" || rm -f $(CONFIG_CLEAN_VPATH_FILES) -rm -f ../vowpalwabbit/$(DEPDIR)/$(am__dirstamp) -rm -f ../vowpalwabbit/$(am__dirstamp) maintainer-clean-generic: @echo "This command is intended for maintainers to use" @echo "it deletes files that may require special tools to rebuild." clean: clean-am clean-am: clean-binPROGRAMS clean-generic clean-libtool mostlyclean-am distclean: distclean-am -rm -rf ../vowpalwabbit/$(DEPDIR) ./$(DEPDIR) -rm -f Makefile distclean-am: clean-am distclean-compile distclean-generic \ distclean-tags dvi: dvi-am dvi-am: html: html-am html-am: info: info-am info-am: install-data-am: install-dvi: install-dvi-am install-dvi-am: install-exec-am: install-binPROGRAMS install-html: install-html-am install-html-am: install-info: install-info-am install-info-am: install-man: install-pdf: install-pdf-am install-pdf-am: install-ps: install-ps-am install-ps-am: installcheck-am: maintainer-clean: maintainer-clean-am -rm -rf ../vowpalwabbit/$(DEPDIR) ./$(DEPDIR) -rm -f Makefile maintainer-clean-am: distclean-am maintainer-clean-generic mostlyclean: mostlyclean-am mostlyclean-am: mostlyclean-compile mostlyclean-generic \ mostlyclean-libtool pdf: pdf-am pdf-am: ps: ps-am ps-am: uninstall-am: uninstall-binPROGRAMS .MAKE: install-am install-strip .PHONY: CTAGS GTAGS TAGS all all-am check check-am clean \ clean-binPROGRAMS clean-generic clean-libtool cscopelist-am \ ctags ctags-am distclean distclean-compile distclean-generic \ distclean-libtool distclean-tags distdir dvi dvi-am html \ html-am info info-am install install-am install-binPROGRAMS \ install-data install-data-am install-dvi install-dvi-am \ install-exec install-exec-am install-html install-html-am \ install-info install-info-am install-man install-pdf \ install-pdf-am install-ps install-ps-am install-strip \ installcheck installcheck-am installdirs maintainer-clean \ maintainer-clean-generic mostlyclean mostlyclean-compile \ mostlyclean-generic mostlyclean-libtool pdf pdf-am ps ps-am \ tags tags-am uninstall uninstall-am uninstall-binPROGRAMS .PRECIOUS: Makefile # Tell versions [3.59,3.63) of GNU make to not export all variables. # Otherwise a system limit (for SysV at least) may be exceeded. .NOEXPORT: vowpal-wabbit-8.5.0.dfsg1/cluster/README_cluster000066400000000000000000000073571323713302100214160ustar00rootroot00000000000000The implementation of Non-linear (Preconditioned) Conjugate Gradient, LBFGS, online learning, and adaptive online learning works on clusters (both Hadoop and otherwise) now. To build the code, run make. At a high level, the code operates by repeatedly executing something equivalent to the MPI AllReduce function---adding up floats from all nodes then broadcasting them back to each individual node. In order to do this, a spanning tree over the nodes must be created. This is done using the helper daemon 'allreduce_master'. *********************************************************************** To run the code on non-Hadoop clusters, the script 'single_machine' has the simplest possible invocation. In general: start the span server on one of the cluster nodes: ./spanning_tree Launch vw on each of the worker nodes: ./vw --span_server --total --node --unique_id -d where: is the host running spanning_tree is the total number of nodes is the node id number is a number shared by all nodes in the process is the input source file for that node *********************************************************************** To run the code on Hadoop clusters: Decide if you are going to control the number of tasks by: (a) using gzip compressed files which cannot be broken up by Hadoop (b) controlling the number of reducers. We'll assume (a) below. Start the span server for the Hadoop cluster: ./spanning_tree Start the map-reduce job using Hadoop streaming: hadoop jar $HADOOP_HOME/hadoop-streaming.jar \ -files vw,runvw.sh \ -Dmapred.job.map.memory.mb=2500 -input -output \ -mapper runvw.sh -reducer NONE where is the directory on HDFS where you want the trained model to be saved. The trained model is saved to the file /model on HDFS and can be retreived by hadoop -get. To modify the arguments to VW, edit the script runvw.sh. Arguments to hadoop can be directly added in the hadoop streaming command. See the 'mapscript.sh' which uses 'runvw.sh' for an advanced example of running VW in a Hadoop enviornmnent. ************************************************************************ The files you need to know about: runvw.sh: This is the mapper code. It takes as arguments: The output directory. The trained model from the first mapper is stored as the file "model" in the output directory. The hostname of the cluster gateway, so that the mappers can connect to the gateway All the other standard VW options are currently hardcoded in the script, feel free to mess around with them. ######################################################################### spanning_tree.cc: This is the span server code which runs on the gateway. You start it before the call to hadoop. The span server backgrounds itself after starting and listens for incoming connections. It sets up the topology on the mappers and then let them communicate amongst themselves. ######################################################################### allreduce.h: This is the header file for the nodes. ######################################################################### allreduce.cc: This is the code for doing allreduce. It implement the routine described above. all_reduce is implemented as a combination of reduce and broadcast routines. reduce reads data from children, adds it with local data and passes it up to the parent with a call to pass_up. broadcast receives data from parent, and passes it down to children with a call to pass_down. ######################################################################### cg.cc, gd.cc, bfgs.cc: learning algorithms which use all_reduce whenever communication is needed. Uses routines accumulate and accumulate_scalar to reduce vectors and scalars resp. vowpal-wabbit-8.5.0.dfsg1/cluster/cluster.vcxproj000077500000000000000000000346101323713302100220660ustar00rootroot00000000000000 DebugLeakCheck Win32 DebugLeakCheck x64 Debug Win32 Debug x64 Release Win32 Release x64 {2720BCD9-6731-4A11-BA24-1F74E35BA97F} Win32Proj cluster spanning_tree ..\vowpalwabbit\ 8.1 ..\sdl\SDL-7.0-Recommended.ruleset true Application true Unicode v140 Application true Unicode v140 Application true Unicode v140 Application true Unicode v140 Application false true Unicode v140 Application false true Unicode v140 true $(Configuration)\$(PlatformShortName)\ $(SolutionDir)\$(PlatformShortName)\$(Configuration)\ true $(Configuration)\$(PlatformShortName)\ $(SolutionDir)\$(PlatformShortName)\$(Configuration)\ true $(SolutionDir)\$(PlatformShortName)\$(Configuration)\ $(SolutionDir)$(PlatformShortName)\$(Configuration)\$(ProjectName)\ true $(SolutionDir)\$(PlatformShortName)\$(Configuration)\ $(SolutionDir)$(PlatformShortName)\$(Configuration)\$(ProjectName)\ false $(Configuration)\$(PlatformShortName)\ $(SolutionDir)\$(PlatformShortName)\$(Configuration)\ false $(SolutionDir)\$(PlatformShortName)\$(Configuration)\ $(SolutionDir)$(PlatformShortName)\$(Configuration)\$(ProjectName)\ Level3 Disabled WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions) $(SolutionDir) Console true ws2_32.lib;%(AdditionalDependencies) xcopy /v /i /r /y "$(SolutionDir)\$(PlatformShortName)\$(Configuration)" "$(SolutionDir)$(Configuration)" Level3 Disabled WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions) $(SolutionDir) Console true ws2_32.lib;%(AdditionalDependencies) xcopy /v /i /r /y "$(SolutionDir)\$(PlatformShortName)\$(Configuration)" "$(SolutionDir)$(Configuration)" Level3 Disabled WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions) $(SolutionDir) MultiThreadedDebug Console true ws2_32.lib;%(AdditionalDependencies) Level3 Disabled WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions) $(SolutionDir) MultiThreadedDebug Console true ws2_32.lib;%(AdditionalDependencies) Level3 MaxSpeed true true WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions) ../vowpalwabbit Console true true true ws2_32.lib;%(AdditionalDependencies) xcopy /v /i /r /y "$(SolutionDir)\$(PlatformShortName)\$(Configuration)" "$(SolutionDir)$(Configuration)" Level3 MaxSpeed true true WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions) MultiThreaded $(SolutionDir) Console true true true ws2_32.lib;%(AdditionalDependencies) This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. vowpal-wabbit-8.5.0.dfsg1/cluster/mapscript-yarn.sh000077500000000000000000000013601323713302100222710ustar00rootroot00000000000000#!/usr/bin/env bash set -e usage="$0 out_dir in_dir" if [ "$2" == "" ] then echo $usage exit fi set -u out_directory=$1 in_directory=$2 hadoop fs -rmr $out_directory > /dev/null 2>&1 || true ./spanning_tree || true hadoop jar /usr/lib/hadoop-mapreduce/hadoop-streaming.jar \ -Dmapred.job.name="vw allreduce $in_directory" \ -Dmapred.map.tasks.speculative.execution=true \ -Dmapred.reduce.tasks=0 \ -Dmapred.child.java.opts="-Xmx100m" \ -Dmapred.task.timeout=600000000 \ -Dmapred.job.map.memory.mb=1000 \ -input $in_directory \ -output $out_directory \ -file ../vowpalwabbit/vw \ -file /usr/lib64/libboost_program_options.so.5 \ -file /lib64/libz.so.1 \ -file runvw-yarn.sh \ -mapper runvw-yarn.sh \ -reducer NONE vowpal-wabbit-8.5.0.dfsg1/cluster/mapscript.sh000077500000000000000000000013761323713302100213310ustar00rootroot00000000000000out_directory=$1 in_directory=$2 nmappers=$3; hadoop fs -rmr $out_directory > /dev/null 2>&1; total=`hadoop fs -ls $in_directory | cut -d " " -f 7 | awk 'BEGIN{sum = 0} {if(NF > 0) sum += $1;} END{print sum;}'` echo $total mapsize=`expr $total / $nmappers` maprem=`expr $total % $nmappers` mapsize=`expr $mapsize + $maprem` mapsize=`expr $mapsize + 100` echo $mapsize ./spanning_tree hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=search -Dmapred.min.split.size=$mapsize -Dmapred.map.tasks.speculative.execution=true -Dmapred.reduce.tasks=0 -Dmapred.job.map.memory.mb=3000 -Dmapred.child.java.opts="-Xmx100m" -Dmapred.task.timeout=600000000 -input $in_directory -output $out_directory -file ../vw -file runvw.sh -mapper runvw.sh -reducer NONE vowpal-wabbit-8.5.0.dfsg1/cluster/runvw-yarn.sh000077500000000000000000000047301323713302100214540ustar00rootroot00000000000000#!/usr/bin/env bash set -e dryrun=$1 # MR1 sets $mapred_map_tasks # MR2/YARN sets $mapreduce_job_maps nmappers=$mapreduce_job_maps # MR1 sets $mapreduce_job_submithost # MR2/YARN sets $mapreduce_job_submithostname submit_host=$mapreduce_job_submithostname # MR1 sets $mapred_output_dir # MR2/YARN sets $mapreduce_output_fileoutputformat_outputdir output_dir=$mapreduce_output_fileoutputformat_outputdir set -u # This works on both MR1 and MR2/YARN mapper=`printenv mapred_task_id | cut -d "_" -f 5` mapred_job_id=`echo "$mapred_job_id" | awk -F "_" '{print $NF}'` # debug echo $mapper > /dev/stderr echo $nmappers > /dev/stderr echo $output_dir > /dev/stderr echo $submit_host > /dev/stderr rm -f temp.cache || true echo 'Starting training' > /dev/stderr # SGD step gdcmd="./vw -b 20 --total $nmappers --node $mapper --unique_id $mapred_job_id --passes 2 --save_per_pass --readable_model sgd.rmodel -d /dev/stdin -f sgd.vwmodel --cache_file temp.cache --span_server $submit_host --loss_function=logistic" # BFGS step mapred_job_id=`expr $mapred_job_id \* 2` #create new nonce bfgscmd="./vw --total $nmappers --node $mapper --unique_id $mapred_job_id --cache_file temp.cache --bfgs --mem 5 --passes 2 --save_per_pass --readable_model bfgs.rmodel --span_server $submit_host -f bfgs.vwmodel -i sgd.vwmodel --loss_function=logistic" if [ "$mapper" == '000000' ] then if [ -z ${dryrun:-} ] then echo "SGD ..." > /dev/stderr $gdcmd > >(tee vw.out) 2> >(tee vw.err >&2) echo "BFGS ..." > /dev/stderr $bfgscmd > >(tee -a vw.out) 2> >(tee -a vw.err >&2) else echo "Dryrrun" echo $gdcmd set cat > /dev/null fi if [ $? -ne 0 ] then exit 5 fi # store models and output in hdfs hadoop fs -put -f sgd.vwmodel* $output_dir || true hadoop fs -put -f sgd.rmodel* $output_dir || true hadoop fs -put -f bfgs.vwmodel* $output_dir || true hadoop fs -put -f bfgs.rmodel* $output_dir || true hadoop fs -put -f vw.* $output_dir || true else if [ -z ${dryrun:-} ] then echo "SGD ..." $gdcmd echo "BFGS ..." $bfgscmd else echo "Dryrrun" echo $gdcmd echo $bfgscmd cat > /dev/null fi if [ $? -ne 0 ] then exit 6 fi fi vowpal-wabbit-8.5.0.dfsg1/cluster/runvw.sh000077500000000000000000000027211323713302100205030ustar00rootroot00000000000000#!/bin/bash mapper=`printenv mapred_task_id | cut -d "_" -f 5` rm -f temp.cache date +"%F %T Start training mapper=$mapper" > /dev/stderr vwcmd="./vw -b 24 --total $mapred_map_tasks --node $mapper --cache_file temp.cache --span_server $mapreduce_job_submithost --loss_function=logistic" mapred_job_id=`echo $mapred_job_id | tr -d 'job_'` gdcmd="$vwcmd --unique_id $mapred_job_id --passes 1 --adaptive --exact_adaptive_norm -d /dev/stdin -f tempmodel" mapred_job_id=`expr $mapred_job_id \* 2` #create new nonce bfgscmd="$vwcmd --unique_id $mapred_job_id --bfgs --mem 5 --passes 20 -f model -i tempmodel" if [ "$mapper" == '000000' ]; then $gdcmd > mapperout 2>&1 if [ $? -ne 0 ]; then date +"%F %T Failed mapper=$mapper cmd=$gdcmd" > /dev/stderr exit 1 fi $bfgscmd >> mapperout 2>&1 outfile=$mapred_output_dir/model mapperfile=$mapred_output_dir/mapperout found=`hadoop fs -lsr | grep $mapred_output_dir | grep mapperout` if [ "$found" != "" ]; then hadoop fs -rm -r $mapperfile fi found=`hadoop fs -lsr | grep $mapred_output_dir | grep model` if [ "$found" != "" ]; then hadoop fs -rm -r $outfile fi date +"%F %T outfile=$outfile" > /dev/stderr hadoop fs -put model $outfile hadoop fs -put mapperout $mapperfile else $gdcmd if [ $? -ne 0 ]; then date +"%F %T Failed mapper=$mapper cmd=$gdcmd" > /dev/stderr exit 1 fi $bfgscmd fi date +"%F %T Done mapper=$mapper" > /dev/stderr vowpal-wabbit-8.5.0.dfsg1/cluster/single_machine000077500000000000000000000011721323713302100216550ustar00rootroot00000000000000#!/bin/sh # A simplest-possible example of parallel code at work on a single machine. ./spanning_tree # This isn't actually useful---to do something useful, you would need to partition the dataset amongst the VW instances head -n 100 ../test/train-sets/0001.dat > head_data tail -n 100 ../test/train-sets/0001.dat > tail_data ../vowpalwabbit/vw --total 2 --node 0 --unique_id 0 -d head_data --cache_file c0 -k --passes 100 --span_server localhost --holdout_off > node_0 2>&1 & ../vowpalwabbit/vw --total 2 --node 1 --unique_id 0 -d tail_data --cache_file c1 -k --passes 100 --span_server localhost --holdout_off killall spanning_tree vowpal-wabbit-8.5.0.dfsg1/cluster/spanning_tree_main.cc000066400000000000000000000024221323713302100231300ustar00rootroot00000000000000/* Copyright (c) 2011 Yahoo! Inc. All rights reserved. The copyrights embodied in the content of this file are licensed under the BSD (revised) open source license This creates a binary tree topology over a set of n nodes that connect. */ #include "spanning_tree.h" #include "vw_exception.h" #ifdef _WIN32 int daemon(int a, int b) { return 0; } int getpid() { return (int) ::GetCurrentProcessId(); } #endif #include #include #include #include #include using namespace std; using namespace VW; int main(int argc, char* argv[]) { if (argc > 2) { cout << "usage: spanning_tree [--nondaemon | pid_file]" << endl; exit(0); } try { if (argc == 2 && strcmp("--nondaemon",argv[1])==0) ; else if (daemon(1,1)) THROWERRNO("daemon: "); SpanningTree spanningTree; if (argc == 2 && strcmp("--nondaemon",argv[1])!=0) { ofstream pid_file; pid_file.open(argv[1]); if (!pid_file.is_open()) { cerr << "error writing pid file" << endl; exit(1); } pid_file << getpid() << endl; pid_file.close(); } spanningTree.Run(); } catch (VW::vw_exception& e) { cerr << "spanning tree (" << e.Filename() << ":" << e.LineNumber() << "): " << e.what() << endl; } } vowpal-wabbit-8.5.0.dfsg1/cluster/spark/000077500000000000000000000000001323713302100201015ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cluster/spark/VwSparkCluster.scala000066400000000000000000000201121323713302100240410ustar00rootroot00000000000000import java.net.InetAddress import org.apache.spark.Logging /** * The goal of this class is to provide as easy method to pipe data through an external command. It is done by combining * a {@link PipedOutputStream} with a {@link PipedInputStream} to create a single pipe to feed data through. This is * done asynchronously so data can be read and written to at the same time. * Created by jmorra on 1/22/15. */ class PipeUtils(bufferSize: Int = 1 << 20) { import java.io._ import scala.concurrent.ExecutionContext.Implicits.global import scala.concurrent.Future import scala.language.postfixOps import scala.sys.process._ /** * This implicit class will allow easy access to streaming through external processes. This * should work on a line by line basis just like Spark's pipe command. * http://stackoverflow.com/questions/28095469/stream-input-to-external-process-in-scala * @param s: The input stream */ implicit class IteratorStream(s: TraversableOnce[String]) { def pipe(cmd: String): Stream[String] = cmd #< iter2is(s) lines def pipe(cmd: Seq[String]): Stream[String] = cmd #< iter2is(s) lines def run(cmd: String): String = cmd #< iter2is(s) !! private[this] def iter2is[A](it: TraversableOnce[A]): InputStream = { // What is written to the output stream will appear in the input stream. val pos = new PipedOutputStream val pis = new PipedInputStream(pos, bufferSize) val w = new PrintWriter(new BufferedOutputStream(pos, bufferSize), false) // Scala 2.11 (scala 2.10, use 'future'). Executes asynchronously. // Fill the stream, then close. Future { try it.foreach(w.println) finally w.close } // Return possibly before pis is fully written to. pis } } } /** * A framework for running VW in a cluster environment using Apache Spark. This * is meant only as a framework and may require some modification to work under your specific case. * Created by jmorra on 8/19/15. */ case class VwSparkCluster( pipeUtils: PipeUtils = new PipeUtils, ipAddress: String = InetAddress.getLocalHost.getHostAddress, defaultParallelism: Int = 2) extends Logging { import java.io._ import org.apache.commons.io.IOUtils import org.apache.spark.rdd.RDD import org.apache.spark.SparkContext import scala.sys.process._ import pipeUtils._ /** * This will learn a VW model in cluster mode. If you notice that this command never starts and just stalls then the parallelism * is probably too high. Refer to this * for more information. * @param data an RDD of Strings that are in VW input format. * @param vwCmd the VW command to run. Note that this command must NOT contain --cache_file and -f. Those will automatically * be appended if necessary. * @param parallelism the amount of parallelism to use. This is calculated using a formula defined in getParallelism * if it is not supplied. It is recommended to only supply this if getParallelism is not working * in you case. * @return a byte array containing the final VW model. */ def train(data: RDD[String], vwCmd: String, parallelism: Option[Int] = None): Array[Byte] = { if (numberOfRunningProcesses("spanning_tree") != 1) { throw new IllegalStateException("spanning_tree is not running on the driver, cannot proceed. Please start spanning_tree and try again.") } val sc = data.context val conf = sc.getConf // By using the job id and the RDD id we should get a globally unique ID. val jobId = (conf.get("spark.app.id").replaceAll("[^\\d]", "") + data.id).toLong logInfo(s"VW cluster job ID: $jobId") val partitions = parallelism.getOrElse(getParallelism(sc).getOrElse(defaultParallelism)) logInfo(s"VW cluster parallelism: ${partitions}") val repartitionedData = if (data.partitions.size == partitions) data else data.repartition(partitions) val vwBaseCmd = s"$vwCmd --total $partitions --span_server $ipAddress --unique_id $jobId" logInfo(s"VW cluster baseCmd: $vwBaseCmd") val vwModels = repartitionedData.mapPartitionsWithIndex{case (partition, x) => Iterator(runVWOnPartition(vwBaseCmd, x, partition)) } vwModels.collect.flatten.flatten } def numberOfRunningProcesses(process: String): Int = "ps aux".#|(s"grep $process").!!.split("\n").size - 1 /** * Gets the executor storage status excluding the driver node. * @param sc the SparkContext * @return an Array of Strings that are the names of all the storage statuses. */ def executors(sc: SparkContext): Array[String] = { sc.getExecutorStorageStatus.collect{ case x if x.blockManagerId.executorId != "" => x.blockManagerId.executorId } } /** * Gets the parallelism of the cluster. This is very much so a work in progress that seems to work now. This took * a lot of experimentation on Spark 1.2.0 to get to work. I make no guarantees that it will work on other Spark versions * especially if dynamic * allocation is enabled. I also only tested this with a master of yarn-client and local so I'm not sure how * well it'll behave in other resource management environments (Spark Standalone, Mesos, etc.). * @param sc the SparkContext * @return if the parallelism can be found then the expected amount of parallelism. */ def getParallelism(sc: SparkContext): Option[Int] = { sc.master match { case x if (x.contains("yarn")) => sc.getConf.getOption("spark.executor.cores").map(x => x.toInt * executors(sc).size) case _ => Some(sc.defaultParallelism) } } /** * This will accept a base VW command, and append a cache file if necessary. It will also create a temp file * to store the VW model. It will then run VW on the supplied data. Finally it will return the bytes of the * model ONLY if the partition is 0. * * This function was tricky to write because the end result of each calculation is a file on the local disk. * According to John all the models should be in the same state after learning so we can choose to save * anyone we want, therefore, transferring the contents of each file to the driver would be wasteful. * In order to avoid this unnecessary transfer we're just going to get the first file. Now you might * ask yourself why not just call .first on the RDD. We cannot do that because in that case Spark would * only evaluate the first mapper and we need all of them to be evaluated, hence the need for .collect to * be called. Note that you may have to increase spark.driver.maxResultSize if the size of the VW model * is too large. * @param vwBaseCmd the base VW command without a cache file or an output specified. A cache file will automatically * be used if --passes is specified. * @param data a String a data in VW format to be passed to VW * @param partition the partition number of this chunk of data * @return an Array of the bytes of the VW model ONLY if this is the 0th partition, else None. */ def runVWOnPartition(vwBaseCmd: String, data: Iterator[String], partition: Int): Option[Array[Byte]] = { val cacheFile = if (vwBaseCmd.contains("--passes ")) { val c = File.createTempFile("vw-cache", ".cache") c.deleteOnExit Option(c) } else None val vwBaseCmdWithCache = cacheFile.map(x => s"$vwBaseCmd -k --cache_file ${x.getCanonicalPath}").getOrElse(vwBaseCmd) val output = File.createTempFile("vw-model", ".model") output.deleteOnExit val vwCmd = s"$vwBaseCmdWithCache --node $partition -f ${output.getCanonicalPath}" data.pipe(vwCmd) cacheFile.foreach(_.delete) val vwModel = if (partition == 0) { val inputStream = new BufferedInputStream(new FileInputStream(output)) val byteArray = IOUtils.toByteArray(inputStream) inputStream.close Option(byteArray) } else None output.delete() vwModel } } vowpal-wabbit-8.5.0.dfsg1/config.guess000077500000000000000000001247531323713302100176340ustar00rootroot00000000000000#! /bin/sh # Attempt to guess a canonical system name. # Copyright 1992-2015 Free Software Foundation, Inc. timestamp='2015-08-20' # This file is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU # General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, see . # # As a special exception to the GNU General Public License, if you # distribute this file as part of a program that contains a # configuration script generated by Autoconf, you may include it under # the same distribution terms that you use for the rest of that # program. This Exception is an additional permission under section 7 # of the GNU General Public License, version 3 ("GPLv3"). # # Originally written by Per Bothner; maintained since 2000 by Ben Elliston. # # You can get the latest version of this script from: # http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD # # Please send patches to . me=`echo "$0" | sed -e 's,.*/,,'` usage="\ Usage: $0 [OPTION] Output the configuration name of the system \`$me' is run on. Operation modes: -h, --help print this help, then exit -t, --time-stamp print date of last modification, then exit -v, --version print version number, then exit Report bugs and patches to ." version="\ GNU config.guess ($timestamp) Originally written by Per Bothner. Copyright 1992-2015 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." help=" Try \`$me --help' for more information." # Parse command line while test $# -gt 0 ; do case $1 in --time-stamp | --time* | -t ) echo "$timestamp" ; exit ;; --version | -v ) echo "$version" ; exit ;; --help | --h* | -h ) echo "$usage"; exit ;; -- ) # Stop option processing shift; break ;; - ) # Use stdin as input. break ;; -* ) echo "$me: invalid option $1$help" >&2 exit 1 ;; * ) break ;; esac done if test $# != 0; then echo "$me: too many arguments$help" >&2 exit 1 fi trap 'exit 1' 1 2 15 # CC_FOR_BUILD -- compiler used by this script. Note that the use of a # compiler to aid in system detection is discouraged as it requires # temporary files to be created and, as you can see below, it is a # headache to deal with in a portable fashion. # Historically, `CC_FOR_BUILD' used to be named `HOST_CC'. We still # use `HOST_CC' if defined, but it is deprecated. # Portable tmp directory creation inspired by the Autoconf team. set_cc_for_build=' trap "exitcode=\$?; (rm -f \$tmpfiles 2>/dev/null; rmdir \$tmp 2>/dev/null) && exit \$exitcode" 0 ; trap "rm -f \$tmpfiles 2>/dev/null; rmdir \$tmp 2>/dev/null; exit 1" 1 2 13 15 ; : ${TMPDIR=/tmp} ; { tmp=`(umask 077 && mktemp -d "$TMPDIR/cgXXXXXX") 2>/dev/null` && test -n "$tmp" && test -d "$tmp" ; } || { test -n "$RANDOM" && tmp=$TMPDIR/cg$$-$RANDOM && (umask 077 && mkdir $tmp) ; } || { tmp=$TMPDIR/cg-$$ && (umask 077 && mkdir $tmp) && echo "Warning: creating insecure temp directory" >&2 ; } || { echo "$me: cannot create a temporary directory in $TMPDIR" >&2 ; exit 1 ; } ; dummy=$tmp/dummy ; tmpfiles="$dummy.c $dummy.o $dummy.rel $dummy" ; case $CC_FOR_BUILD,$HOST_CC,$CC in ,,) echo "int x;" > $dummy.c ; for c in cc gcc c89 c99 ; do if ($c -c -o $dummy.o $dummy.c) >/dev/null 2>&1 ; then CC_FOR_BUILD="$c"; break ; fi ; done ; if test x"$CC_FOR_BUILD" = x ; then CC_FOR_BUILD=no_compiler_found ; fi ;; ,,*) CC_FOR_BUILD=$CC ;; ,*,*) CC_FOR_BUILD=$HOST_CC ;; esac ; set_cc_for_build= ;' # This is needed to find uname on a Pyramid OSx when run in the BSD universe. # (ghazi@noc.rutgers.edu 1994-08-24) if (test -f /.attbin/uname) >/dev/null 2>&1 ; then PATH=$PATH:/.attbin ; export PATH fi UNAME_MACHINE=`(uname -m) 2>/dev/null` || UNAME_MACHINE=unknown UNAME_RELEASE=`(uname -r) 2>/dev/null` || UNAME_RELEASE=unknown UNAME_SYSTEM=`(uname -s) 2>/dev/null` || UNAME_SYSTEM=unknown UNAME_VERSION=`(uname -v) 2>/dev/null` || UNAME_VERSION=unknown case "${UNAME_SYSTEM}" in Linux|GNU|GNU/*) # If the system lacks a compiler, then just pick glibc. # We could probably try harder. LIBC=gnu eval $set_cc_for_build cat <<-EOF > $dummy.c #include #if defined(__UCLIBC__) LIBC=uclibc #elif defined(__dietlibc__) LIBC=dietlibc #else LIBC=gnu #endif EOF eval `$CC_FOR_BUILD -E $dummy.c 2>/dev/null | grep '^LIBC' | sed 's, ,,g'` ;; esac # Note: order is significant - the case branches are not exclusive. case "${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE}:${UNAME_VERSION}" in *:NetBSD:*:*) # NetBSD (nbsd) targets should (where applicable) match one or # more of the tuples: *-*-netbsdelf*, *-*-netbsdaout*, # *-*-netbsdecoff* and *-*-netbsd*. For targets that recently # switched to ELF, *-*-netbsd* would select the old # object file format. This provides both forward # compatibility and a consistent mechanism for selecting the # object file format. # # Note: NetBSD doesn't particularly care about the vendor # portion of the name. We always set it to "unknown". sysctl="sysctl -n hw.machine_arch" UNAME_MACHINE_ARCH=`(uname -p 2>/dev/null || \ /sbin/$sysctl 2>/dev/null || \ /usr/sbin/$sysctl 2>/dev/null || \ echo unknown)` case "${UNAME_MACHINE_ARCH}" in armeb) machine=armeb-unknown ;; arm*) machine=arm-unknown ;; sh3el) machine=shl-unknown ;; sh3eb) machine=sh-unknown ;; sh5el) machine=sh5le-unknown ;; earmv*) arch=`echo ${UNAME_MACHINE_ARCH} | sed -e 's,^e\(armv[0-9]\).*$,\1,'` endian=`echo ${UNAME_MACHINE_ARCH} | sed -ne 's,^.*\(eb\)$,\1,p'` machine=${arch}${endian}-unknown ;; *) machine=${UNAME_MACHINE_ARCH}-unknown ;; esac # The Operating System including object format, if it has switched # to ELF recently, or will in the future. case "${UNAME_MACHINE_ARCH}" in arm*|earm*|i386|m68k|ns32k|sh3*|sparc|vax) eval $set_cc_for_build if echo __ELF__ | $CC_FOR_BUILD -E - 2>/dev/null \ | grep -q __ELF__ then # Once all utilities can be ECOFF (netbsdecoff) or a.out (netbsdaout). # Return netbsd for either. FIX? os=netbsd else os=netbsdelf fi ;; *) os=netbsd ;; esac # Determine ABI tags. case "${UNAME_MACHINE_ARCH}" in earm*) expr='s/^earmv[0-9]/-eabi/;s/eb$//' abi=`echo ${UNAME_MACHINE_ARCH} | sed -e "$expr"` ;; esac # The OS release # Debian GNU/NetBSD machines have a different userland, and # thus, need a distinct triplet. However, they do not need # kernel version information, so it can be replaced with a # suitable tag, in the style of linux-gnu. case "${UNAME_VERSION}" in Debian*) release='-gnu' ;; *) release=`echo ${UNAME_RELEASE} | sed -e 's/[-_].*//' | cut -d. -f1,2` ;; esac # Since CPU_TYPE-MANUFACTURER-KERNEL-OPERATING_SYSTEM: # contains redundant information, the shorter form: # CPU_TYPE-MANUFACTURER-OPERATING_SYSTEM is used. echo "${machine}-${os}${release}${abi}" exit ;; *:Bitrig:*:*) UNAME_MACHINE_ARCH=`arch | sed 's/Bitrig.//'` echo ${UNAME_MACHINE_ARCH}-unknown-bitrig${UNAME_RELEASE} exit ;; *:OpenBSD:*:*) UNAME_MACHINE_ARCH=`arch | sed 's/OpenBSD.//'` echo ${UNAME_MACHINE_ARCH}-unknown-openbsd${UNAME_RELEASE} exit ;; *:ekkoBSD:*:*) echo ${UNAME_MACHINE}-unknown-ekkobsd${UNAME_RELEASE} exit ;; *:SolidBSD:*:*) echo ${UNAME_MACHINE}-unknown-solidbsd${UNAME_RELEASE} exit ;; macppc:MirBSD:*:*) echo powerpc-unknown-mirbsd${UNAME_RELEASE} exit ;; *:MirBSD:*:*) echo ${UNAME_MACHINE}-unknown-mirbsd${UNAME_RELEASE} exit ;; *:Sortix:*:*) echo ${UNAME_MACHINE}-unknown-sortix exit ;; alpha:OSF1:*:*) case $UNAME_RELEASE in *4.0) UNAME_RELEASE=`/usr/sbin/sizer -v | awk '{print $3}'` ;; *5.*) UNAME_RELEASE=`/usr/sbin/sizer -v | awk '{print $4}'` ;; esac # According to Compaq, /usr/sbin/psrinfo has been available on # OSF/1 and Tru64 systems produced since 1995. I hope that # covers most systems running today. This code pipes the CPU # types through head -n 1, so we only detect the type of CPU 0. ALPHA_CPU_TYPE=`/usr/sbin/psrinfo -v | sed -n -e 's/^ The alpha \(.*\) processor.*$/\1/p' | head -n 1` case "$ALPHA_CPU_TYPE" in "EV4 (21064)") UNAME_MACHINE="alpha" ;; "EV4.5 (21064)") UNAME_MACHINE="alpha" ;; "LCA4 (21066/21068)") UNAME_MACHINE="alpha" ;; "EV5 (21164)") UNAME_MACHINE="alphaev5" ;; "EV5.6 (21164A)") UNAME_MACHINE="alphaev56" ;; "EV5.6 (21164PC)") UNAME_MACHINE="alphapca56" ;; "EV5.7 (21164PC)") UNAME_MACHINE="alphapca57" ;; "EV6 (21264)") UNAME_MACHINE="alphaev6" ;; "EV6.7 (21264A)") UNAME_MACHINE="alphaev67" ;; "EV6.8CB (21264C)") UNAME_MACHINE="alphaev68" ;; "EV6.8AL (21264B)") UNAME_MACHINE="alphaev68" ;; "EV6.8CX (21264D)") UNAME_MACHINE="alphaev68" ;; "EV6.9A (21264/EV69A)") UNAME_MACHINE="alphaev69" ;; "EV7 (21364)") UNAME_MACHINE="alphaev7" ;; "EV7.9 (21364A)") UNAME_MACHINE="alphaev79" ;; esac # A Pn.n version is a patched version. # A Vn.n version is a released version. # A Tn.n version is a released field test version. # A Xn.n version is an unreleased experimental baselevel. # 1.2 uses "1.2" for uname -r. echo ${UNAME_MACHINE}-dec-osf`echo ${UNAME_RELEASE} | sed -e 's/^[PVTX]//' | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` # Reset EXIT trap before exiting to avoid spurious non-zero exit code. exitcode=$? trap '' 0 exit $exitcode ;; Alpha\ *:Windows_NT*:*) # How do we know it's Interix rather than the generic POSIX subsystem? # Should we change UNAME_MACHINE based on the output of uname instead # of the specific Alpha model? echo alpha-pc-interix exit ;; 21064:Windows_NT:50:3) echo alpha-dec-winnt3.5 exit ;; Amiga*:UNIX_System_V:4.0:*) echo m68k-unknown-sysv4 exit ;; *:[Aa]miga[Oo][Ss]:*:*) echo ${UNAME_MACHINE}-unknown-amigaos exit ;; *:[Mm]orph[Oo][Ss]:*:*) echo ${UNAME_MACHINE}-unknown-morphos exit ;; *:OS/390:*:*) echo i370-ibm-openedition exit ;; *:z/VM:*:*) echo s390-ibm-zvmoe exit ;; *:OS400:*:*) echo powerpc-ibm-os400 exit ;; arm:RISC*:1.[012]*:*|arm:riscix:1.[012]*:*) echo arm-acorn-riscix${UNAME_RELEASE} exit ;; arm*:riscos:*:*|arm*:RISCOS:*:*) echo arm-unknown-riscos exit ;; SR2?01:HI-UX/MPP:*:* | SR8000:HI-UX/MPP:*:*) echo hppa1.1-hitachi-hiuxmpp exit ;; Pyramid*:OSx*:*:* | MIS*:OSx*:*:* | MIS*:SMP_DC-OSx*:*:*) # akee@wpdis03.wpafb.af.mil (Earle F. Ake) contributed MIS and NILE. if test "`(/bin/universe) 2>/dev/null`" = att ; then echo pyramid-pyramid-sysv3 else echo pyramid-pyramid-bsd fi exit ;; NILE*:*:*:dcosx) echo pyramid-pyramid-svr4 exit ;; DRS?6000:unix:4.0:6*) echo sparc-icl-nx6 exit ;; DRS?6000:UNIX_SV:4.2*:7* | DRS?6000:isis:4.2*:7*) case `/usr/bin/uname -p` in sparc) echo sparc-icl-nx7; exit ;; esac ;; s390x:SunOS:*:*) echo ${UNAME_MACHINE}-ibm-solaris2`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'` exit ;; sun4H:SunOS:5.*:*) echo sparc-hal-solaris2`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'` exit ;; sun4*:SunOS:5.*:* | tadpole*:SunOS:5.*:*) echo sparc-sun-solaris2`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'` exit ;; i86pc:AuroraUX:5.*:* | i86xen:AuroraUX:5.*:*) echo i386-pc-auroraux${UNAME_RELEASE} exit ;; i86pc:SunOS:5.*:* | i86xen:SunOS:5.*:*) eval $set_cc_for_build SUN_ARCH="i386" # If there is a compiler, see if it is configured for 64-bit objects. # Note that the Sun cc does not turn __LP64__ into 1 like gcc does. # This test works for both compilers. if [ "$CC_FOR_BUILD" != 'no_compiler_found' ]; then if (echo '#ifdef __amd64'; echo IS_64BIT_ARCH; echo '#endif') | \ (CCOPTS= $CC_FOR_BUILD -E - 2>/dev/null) | \ grep IS_64BIT_ARCH >/dev/null then SUN_ARCH="x86_64" fi fi echo ${SUN_ARCH}-pc-solaris2`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'` exit ;; sun4*:SunOS:6*:*) # According to config.sub, this is the proper way to canonicalize # SunOS6. Hard to guess exactly what SunOS6 will be like, but # it's likely to be more like Solaris than SunOS4. echo sparc-sun-solaris3`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'` exit ;; sun4*:SunOS:*:*) case "`/usr/bin/arch -k`" in Series*|S4*) UNAME_RELEASE=`uname -v` ;; esac # Japanese Language versions have a version number like `4.1.3-JL'. echo sparc-sun-sunos`echo ${UNAME_RELEASE}|sed -e 's/-/_/'` exit ;; sun3*:SunOS:*:*) echo m68k-sun-sunos${UNAME_RELEASE} exit ;; sun*:*:4.2BSD:*) UNAME_RELEASE=`(sed 1q /etc/motd | awk '{print substr($5,1,3)}') 2>/dev/null` test "x${UNAME_RELEASE}" = "x" && UNAME_RELEASE=3 case "`/bin/arch`" in sun3) echo m68k-sun-sunos${UNAME_RELEASE} ;; sun4) echo sparc-sun-sunos${UNAME_RELEASE} ;; esac exit ;; aushp:SunOS:*:*) echo sparc-auspex-sunos${UNAME_RELEASE} exit ;; # The situation for MiNT is a little confusing. The machine name # can be virtually everything (everything which is not # "atarist" or "atariste" at least should have a processor # > m68000). The system name ranges from "MiNT" over "FreeMiNT" # to the lowercase version "mint" (or "freemint"). Finally # the system name "TOS" denotes a system which is actually not # MiNT. But MiNT is downward compatible to TOS, so this should # be no problem. atarist[e]:*MiNT:*:* | atarist[e]:*mint:*:* | atarist[e]:*TOS:*:*) echo m68k-atari-mint${UNAME_RELEASE} exit ;; atari*:*MiNT:*:* | atari*:*mint:*:* | atarist[e]:*TOS:*:*) echo m68k-atari-mint${UNAME_RELEASE} exit ;; *falcon*:*MiNT:*:* | *falcon*:*mint:*:* | *falcon*:*TOS:*:*) echo m68k-atari-mint${UNAME_RELEASE} exit ;; milan*:*MiNT:*:* | milan*:*mint:*:* | *milan*:*TOS:*:*) echo m68k-milan-mint${UNAME_RELEASE} exit ;; hades*:*MiNT:*:* | hades*:*mint:*:* | *hades*:*TOS:*:*) echo m68k-hades-mint${UNAME_RELEASE} exit ;; *:*MiNT:*:* | *:*mint:*:* | *:*TOS:*:*) echo m68k-unknown-mint${UNAME_RELEASE} exit ;; m68k:machten:*:*) echo m68k-apple-machten${UNAME_RELEASE} exit ;; powerpc:machten:*:*) echo powerpc-apple-machten${UNAME_RELEASE} exit ;; RISC*:Mach:*:*) echo mips-dec-mach_bsd4.3 exit ;; RISC*:ULTRIX:*:*) echo mips-dec-ultrix${UNAME_RELEASE} exit ;; VAX*:ULTRIX*:*:*) echo vax-dec-ultrix${UNAME_RELEASE} exit ;; 2020:CLIX:*:* | 2430:CLIX:*:*) echo clipper-intergraph-clix${UNAME_RELEASE} exit ;; mips:*:*:UMIPS | mips:*:*:RISCos) eval $set_cc_for_build sed 's/^ //' << EOF >$dummy.c #ifdef __cplusplus #include /* for printf() prototype */ int main (int argc, char *argv[]) { #else int main (argc, argv) int argc; char *argv[]; { #endif #if defined (host_mips) && defined (MIPSEB) #if defined (SYSTYPE_SYSV) printf ("mips-mips-riscos%ssysv\n", argv[1]); exit (0); #endif #if defined (SYSTYPE_SVR4) printf ("mips-mips-riscos%ssvr4\n", argv[1]); exit (0); #endif #if defined (SYSTYPE_BSD43) || defined(SYSTYPE_BSD) printf ("mips-mips-riscos%sbsd\n", argv[1]); exit (0); #endif #endif exit (-1); } EOF $CC_FOR_BUILD -o $dummy $dummy.c && dummyarg=`echo "${UNAME_RELEASE}" | sed -n 's/\([0-9]*\).*/\1/p'` && SYSTEM_NAME=`$dummy $dummyarg` && { echo "$SYSTEM_NAME"; exit; } echo mips-mips-riscos${UNAME_RELEASE} exit ;; Motorola:PowerMAX_OS:*:*) echo powerpc-motorola-powermax exit ;; Motorola:*:4.3:PL8-*) echo powerpc-harris-powermax exit ;; Night_Hawk:*:*:PowerMAX_OS | Synergy:PowerMAX_OS:*:*) echo powerpc-harris-powermax exit ;; Night_Hawk:Power_UNIX:*:*) echo powerpc-harris-powerunix exit ;; m88k:CX/UX:7*:*) echo m88k-harris-cxux7 exit ;; m88k:*:4*:R4*) echo m88k-motorola-sysv4 exit ;; m88k:*:3*:R3*) echo m88k-motorola-sysv3 exit ;; AViiON:dgux:*:*) # DG/UX returns AViiON for all architectures UNAME_PROCESSOR=`/usr/bin/uname -p` if [ $UNAME_PROCESSOR = mc88100 ] || [ $UNAME_PROCESSOR = mc88110 ] then if [ ${TARGET_BINARY_INTERFACE}x = m88kdguxelfx ] || \ [ ${TARGET_BINARY_INTERFACE}x = x ] then echo m88k-dg-dgux${UNAME_RELEASE} else echo m88k-dg-dguxbcs${UNAME_RELEASE} fi else echo i586-dg-dgux${UNAME_RELEASE} fi exit ;; M88*:DolphinOS:*:*) # DolphinOS (SVR3) echo m88k-dolphin-sysv3 exit ;; M88*:*:R3*:*) # Delta 88k system running SVR3 echo m88k-motorola-sysv3 exit ;; XD88*:*:*:*) # Tektronix XD88 system running UTekV (SVR3) echo m88k-tektronix-sysv3 exit ;; Tek43[0-9][0-9]:UTek:*:*) # Tektronix 4300 system running UTek (BSD) echo m68k-tektronix-bsd exit ;; *:IRIX*:*:*) echo mips-sgi-irix`echo ${UNAME_RELEASE}|sed -e 's/-/_/g'` exit ;; ????????:AIX?:[12].1:2) # AIX 2.2.1 or AIX 2.1.1 is RT/PC AIX. echo romp-ibm-aix # uname -m gives an 8 hex-code CPU id exit ;; # Note that: echo "'`uname -s`'" gives 'AIX ' i*86:AIX:*:*) echo i386-ibm-aix exit ;; ia64:AIX:*:*) if [ -x /usr/bin/oslevel ] ; then IBM_REV=`/usr/bin/oslevel` else IBM_REV=${UNAME_VERSION}.${UNAME_RELEASE} fi echo ${UNAME_MACHINE}-ibm-aix${IBM_REV} exit ;; *:AIX:2:3) if grep bos325 /usr/include/stdio.h >/dev/null 2>&1; then eval $set_cc_for_build sed 's/^ //' << EOF >$dummy.c #include main() { if (!__power_pc()) exit(1); puts("powerpc-ibm-aix3.2.5"); exit(0); } EOF if $CC_FOR_BUILD -o $dummy $dummy.c && SYSTEM_NAME=`$dummy` then echo "$SYSTEM_NAME" else echo rs6000-ibm-aix3.2.5 fi elif grep bos324 /usr/include/stdio.h >/dev/null 2>&1; then echo rs6000-ibm-aix3.2.4 else echo rs6000-ibm-aix3.2 fi exit ;; *:AIX:*:[4567]) IBM_CPU_ID=`/usr/sbin/lsdev -C -c processor -S available | sed 1q | awk '{ print $1 }'` if /usr/sbin/lsattr -El ${IBM_CPU_ID} | grep ' POWER' >/dev/null 2>&1; then IBM_ARCH=rs6000 else IBM_ARCH=powerpc fi if [ -x /usr/bin/lslpp ] ; then IBM_REV=`/usr/bin/lslpp -Lqc bos.rte.libc | awk -F: '{ print $3 }' | sed s/[0-9]*$/0/` else IBM_REV=${UNAME_VERSION}.${UNAME_RELEASE} fi echo ${IBM_ARCH}-ibm-aix${IBM_REV} exit ;; *:AIX:*:*) echo rs6000-ibm-aix exit ;; ibmrt:4.4BSD:*|romp-ibm:BSD:*) echo romp-ibm-bsd4.4 exit ;; ibmrt:*BSD:*|romp-ibm:BSD:*) # covers RT/PC BSD and echo romp-ibm-bsd${UNAME_RELEASE} # 4.3 with uname added to exit ;; # report: romp-ibm BSD 4.3 *:BOSX:*:*) echo rs6000-bull-bosx exit ;; DPX/2?00:B.O.S.:*:*) echo m68k-bull-sysv3 exit ;; 9000/[34]??:4.3bsd:1.*:*) echo m68k-hp-bsd exit ;; hp300:4.4BSD:*:* | 9000/[34]??:4.3bsd:2.*:*) echo m68k-hp-bsd4.4 exit ;; 9000/[34678]??:HP-UX:*:*) HPUX_REV=`echo ${UNAME_RELEASE}|sed -e 's/[^.]*.[0B]*//'` case "${UNAME_MACHINE}" in 9000/31? ) HP_ARCH=m68000 ;; 9000/[34]?? ) HP_ARCH=m68k ;; 9000/[678][0-9][0-9]) if [ -x /usr/bin/getconf ]; then sc_cpu_version=`/usr/bin/getconf SC_CPU_VERSION 2>/dev/null` sc_kernel_bits=`/usr/bin/getconf SC_KERNEL_BITS 2>/dev/null` case "${sc_cpu_version}" in 523) HP_ARCH="hppa1.0" ;; # CPU_PA_RISC1_0 528) HP_ARCH="hppa1.1" ;; # CPU_PA_RISC1_1 532) # CPU_PA_RISC2_0 case "${sc_kernel_bits}" in 32) HP_ARCH="hppa2.0n" ;; 64) HP_ARCH="hppa2.0w" ;; '') HP_ARCH="hppa2.0" ;; # HP-UX 10.20 esac ;; esac fi if [ "${HP_ARCH}" = "" ]; then eval $set_cc_for_build sed 's/^ //' << EOF >$dummy.c #define _HPUX_SOURCE #include #include int main () { #if defined(_SC_KERNEL_BITS) long bits = sysconf(_SC_KERNEL_BITS); #endif long cpu = sysconf (_SC_CPU_VERSION); switch (cpu) { case CPU_PA_RISC1_0: puts ("hppa1.0"); break; case CPU_PA_RISC1_1: puts ("hppa1.1"); break; case CPU_PA_RISC2_0: #if defined(_SC_KERNEL_BITS) switch (bits) { case 64: puts ("hppa2.0w"); break; case 32: puts ("hppa2.0n"); break; default: puts ("hppa2.0"); break; } break; #else /* !defined(_SC_KERNEL_BITS) */ puts ("hppa2.0"); break; #endif default: puts ("hppa1.0"); break; } exit (0); } EOF (CCOPTS= $CC_FOR_BUILD -o $dummy $dummy.c 2>/dev/null) && HP_ARCH=`$dummy` test -z "$HP_ARCH" && HP_ARCH=hppa fi ;; esac if [ ${HP_ARCH} = "hppa2.0w" ] then eval $set_cc_for_build # hppa2.0w-hp-hpux* has a 64-bit kernel and a compiler generating # 32-bit code. hppa64-hp-hpux* has the same kernel and a compiler # generating 64-bit code. GNU and HP use different nomenclature: # # $ CC_FOR_BUILD=cc ./config.guess # => hppa2.0w-hp-hpux11.23 # $ CC_FOR_BUILD="cc +DA2.0w" ./config.guess # => hppa64-hp-hpux11.23 if echo __LP64__ | (CCOPTS= $CC_FOR_BUILD -E - 2>/dev/null) | grep -q __LP64__ then HP_ARCH="hppa2.0w" else HP_ARCH="hppa64" fi fi echo ${HP_ARCH}-hp-hpux${HPUX_REV} exit ;; ia64:HP-UX:*:*) HPUX_REV=`echo ${UNAME_RELEASE}|sed -e 's/[^.]*.[0B]*//'` echo ia64-hp-hpux${HPUX_REV} exit ;; 3050*:HI-UX:*:*) eval $set_cc_for_build sed 's/^ //' << EOF >$dummy.c #include int main () { long cpu = sysconf (_SC_CPU_VERSION); /* The order matters, because CPU_IS_HP_MC68K erroneously returns true for CPU_PA_RISC1_0. CPU_IS_PA_RISC returns correct results, however. */ if (CPU_IS_PA_RISC (cpu)) { switch (cpu) { case CPU_PA_RISC1_0: puts ("hppa1.0-hitachi-hiuxwe2"); break; case CPU_PA_RISC1_1: puts ("hppa1.1-hitachi-hiuxwe2"); break; case CPU_PA_RISC2_0: puts ("hppa2.0-hitachi-hiuxwe2"); break; default: puts ("hppa-hitachi-hiuxwe2"); break; } } else if (CPU_IS_HP_MC68K (cpu)) puts ("m68k-hitachi-hiuxwe2"); else puts ("unknown-hitachi-hiuxwe2"); exit (0); } EOF $CC_FOR_BUILD -o $dummy $dummy.c && SYSTEM_NAME=`$dummy` && { echo "$SYSTEM_NAME"; exit; } echo unknown-hitachi-hiuxwe2 exit ;; 9000/7??:4.3bsd:*:* | 9000/8?[79]:4.3bsd:*:* ) echo hppa1.1-hp-bsd exit ;; 9000/8??:4.3bsd:*:*) echo hppa1.0-hp-bsd exit ;; *9??*:MPE/iX:*:* | *3000*:MPE/iX:*:*) echo hppa1.0-hp-mpeix exit ;; hp7??:OSF1:*:* | hp8?[79]:OSF1:*:* ) echo hppa1.1-hp-osf exit ;; hp8??:OSF1:*:*) echo hppa1.0-hp-osf exit ;; i*86:OSF1:*:*) if [ -x /usr/sbin/sysversion ] ; then echo ${UNAME_MACHINE}-unknown-osf1mk else echo ${UNAME_MACHINE}-unknown-osf1 fi exit ;; parisc*:Lites*:*:*) echo hppa1.1-hp-lites exit ;; C1*:ConvexOS:*:* | convex:ConvexOS:C1*:*) echo c1-convex-bsd exit ;; C2*:ConvexOS:*:* | convex:ConvexOS:C2*:*) if getsysinfo -f scalar_acc then echo c32-convex-bsd else echo c2-convex-bsd fi exit ;; C34*:ConvexOS:*:* | convex:ConvexOS:C34*:*) echo c34-convex-bsd exit ;; C38*:ConvexOS:*:* | convex:ConvexOS:C38*:*) echo c38-convex-bsd exit ;; C4*:ConvexOS:*:* | convex:ConvexOS:C4*:*) echo c4-convex-bsd exit ;; CRAY*Y-MP:*:*:*) echo ymp-cray-unicos${UNAME_RELEASE} | sed -e 's/\.[^.]*$/.X/' exit ;; CRAY*[A-Z]90:*:*:*) echo ${UNAME_MACHINE}-cray-unicos${UNAME_RELEASE} \ | sed -e 's/CRAY.*\([A-Z]90\)/\1/' \ -e y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/ \ -e 's/\.[^.]*$/.X/' exit ;; CRAY*TS:*:*:*) echo t90-cray-unicos${UNAME_RELEASE} | sed -e 's/\.[^.]*$/.X/' exit ;; CRAY*T3E:*:*:*) echo alphaev5-cray-unicosmk${UNAME_RELEASE} | sed -e 's/\.[^.]*$/.X/' exit ;; CRAY*SV1:*:*:*) echo sv1-cray-unicos${UNAME_RELEASE} | sed -e 's/\.[^.]*$/.X/' exit ;; *:UNICOS/mp:*:*) echo craynv-cray-unicosmp${UNAME_RELEASE} | sed -e 's/\.[^.]*$/.X/' exit ;; F30[01]:UNIX_System_V:*:* | F700:UNIX_System_V:*:*) FUJITSU_PROC=`uname -m | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` FUJITSU_SYS=`uname -p | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz' | sed -e 's/\///'` FUJITSU_REL=`echo ${UNAME_RELEASE} | sed -e 's/ /_/'` echo "${FUJITSU_PROC}-fujitsu-${FUJITSU_SYS}${FUJITSU_REL}" exit ;; 5000:UNIX_System_V:4.*:*) FUJITSU_SYS=`uname -p | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz' | sed -e 's/\///'` FUJITSU_REL=`echo ${UNAME_RELEASE} | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz' | sed -e 's/ /_/'` echo "sparc-fujitsu-${FUJITSU_SYS}${FUJITSU_REL}" exit ;; i*86:BSD/386:*:* | i*86:BSD/OS:*:* | *:Ascend\ Embedded/OS:*:*) echo ${UNAME_MACHINE}-pc-bsdi${UNAME_RELEASE} exit ;; sparc*:BSD/OS:*:*) echo sparc-unknown-bsdi${UNAME_RELEASE} exit ;; *:BSD/OS:*:*) echo ${UNAME_MACHINE}-unknown-bsdi${UNAME_RELEASE} exit ;; *:FreeBSD:*:*) UNAME_PROCESSOR=`/usr/bin/uname -p` case ${UNAME_PROCESSOR} in amd64) echo x86_64-unknown-freebsd`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'` ;; *) echo ${UNAME_PROCESSOR}-unknown-freebsd`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'` ;; esac exit ;; i*:CYGWIN*:*) echo ${UNAME_MACHINE}-pc-cygwin exit ;; *:MINGW64*:*) echo ${UNAME_MACHINE}-pc-mingw64 exit ;; *:MINGW*:*) echo ${UNAME_MACHINE}-pc-mingw32 exit ;; *:MSYS*:*) echo ${UNAME_MACHINE}-pc-msys exit ;; i*:windows32*:*) # uname -m includes "-pc" on this system. echo ${UNAME_MACHINE}-mingw32 exit ;; i*:PW*:*) echo ${UNAME_MACHINE}-pc-pw32 exit ;; *:Interix*:*) case ${UNAME_MACHINE} in x86) echo i586-pc-interix${UNAME_RELEASE} exit ;; authenticamd | genuineintel | EM64T) echo x86_64-unknown-interix${UNAME_RELEASE} exit ;; IA64) echo ia64-unknown-interix${UNAME_RELEASE} exit ;; esac ;; [345]86:Windows_95:* | [345]86:Windows_98:* | [345]86:Windows_NT:*) echo i${UNAME_MACHINE}-pc-mks exit ;; 8664:Windows_NT:*) echo x86_64-pc-mks exit ;; i*:Windows_NT*:* | Pentium*:Windows_NT*:*) # How do we know it's Interix rather than the generic POSIX subsystem? # It also conflicts with pre-2.0 versions of AT&T UWIN. Should we # UNAME_MACHINE based on the output of uname instead of i386? echo i586-pc-interix exit ;; i*:UWIN*:*) echo ${UNAME_MACHINE}-pc-uwin exit ;; amd64:CYGWIN*:*:* | x86_64:CYGWIN*:*:*) echo x86_64-unknown-cygwin exit ;; p*:CYGWIN*:*) echo powerpcle-unknown-cygwin exit ;; prep*:SunOS:5.*:*) echo powerpcle-unknown-solaris2`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'` exit ;; *:GNU:*:*) # the GNU system echo `echo ${UNAME_MACHINE}|sed -e 's,[-/].*$,,'`-unknown-${LIBC}`echo ${UNAME_RELEASE}|sed -e 's,/.*$,,'` exit ;; *:GNU/*:*:*) # other systems with GNU libc and userland echo ${UNAME_MACHINE}-unknown-`echo ${UNAME_SYSTEM} | sed 's,^[^/]*/,,' | tr '[A-Z]' '[a-z]'``echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'`-${LIBC} exit ;; i*86:Minix:*:*) echo ${UNAME_MACHINE}-pc-minix exit ;; aarch64:Linux:*:*) echo ${UNAME_MACHINE}-unknown-linux-${LIBC} exit ;; aarch64_be:Linux:*:*) UNAME_MACHINE=aarch64_be echo ${UNAME_MACHINE}-unknown-linux-${LIBC} exit ;; alpha:Linux:*:*) case `sed -n '/^cpu model/s/^.*: \(.*\)/\1/p' < /proc/cpuinfo` in EV5) UNAME_MACHINE=alphaev5 ;; EV56) UNAME_MACHINE=alphaev56 ;; PCA56) UNAME_MACHINE=alphapca56 ;; PCA57) UNAME_MACHINE=alphapca56 ;; EV6) UNAME_MACHINE=alphaev6 ;; EV67) UNAME_MACHINE=alphaev67 ;; EV68*) UNAME_MACHINE=alphaev68 ;; esac objdump --private-headers /bin/sh | grep -q ld.so.1 if test "$?" = 0 ; then LIBC="gnulibc1" ; fi echo ${UNAME_MACHINE}-unknown-linux-${LIBC} exit ;; arc:Linux:*:* | arceb:Linux:*:*) echo ${UNAME_MACHINE}-unknown-linux-${LIBC} exit ;; arm*:Linux:*:*) eval $set_cc_for_build if echo __ARM_EABI__ | $CC_FOR_BUILD -E - 2>/dev/null \ | grep -q __ARM_EABI__ then echo ${UNAME_MACHINE}-unknown-linux-${LIBC} else if echo __ARM_PCS_VFP | $CC_FOR_BUILD -E - 2>/dev/null \ | grep -q __ARM_PCS_VFP then echo ${UNAME_MACHINE}-unknown-linux-${LIBC}eabi else echo ${UNAME_MACHINE}-unknown-linux-${LIBC}eabihf fi fi exit ;; avr32*:Linux:*:*) echo ${UNAME_MACHINE}-unknown-linux-${LIBC} exit ;; cris:Linux:*:*) echo ${UNAME_MACHINE}-axis-linux-${LIBC} exit ;; crisv32:Linux:*:*) echo ${UNAME_MACHINE}-axis-linux-${LIBC} exit ;; e2k:Linux:*:*) echo ${UNAME_MACHINE}-unknown-linux-${LIBC} exit ;; frv:Linux:*:*) echo ${UNAME_MACHINE}-unknown-linux-${LIBC} exit ;; hexagon:Linux:*:*) echo ${UNAME_MACHINE}-unknown-linux-${LIBC} exit ;; i*86:Linux:*:*) echo ${UNAME_MACHINE}-pc-linux-${LIBC} exit ;; ia64:Linux:*:*) echo ${UNAME_MACHINE}-unknown-linux-${LIBC} exit ;; m32r*:Linux:*:*) echo ${UNAME_MACHINE}-unknown-linux-${LIBC} exit ;; m68*:Linux:*:*) echo ${UNAME_MACHINE}-unknown-linux-${LIBC} exit ;; mips:Linux:*:* | mips64:Linux:*:*) eval $set_cc_for_build sed 's/^ //' << EOF >$dummy.c #undef CPU #undef ${UNAME_MACHINE} #undef ${UNAME_MACHINE}el #if defined(__MIPSEL__) || defined(__MIPSEL) || defined(_MIPSEL) || defined(MIPSEL) CPU=${UNAME_MACHINE}el #else #if defined(__MIPSEB__) || defined(__MIPSEB) || defined(_MIPSEB) || defined(MIPSEB) CPU=${UNAME_MACHINE} #else CPU= #endif #endif EOF eval `$CC_FOR_BUILD -E $dummy.c 2>/dev/null | grep '^CPU'` test x"${CPU}" != x && { echo "${CPU}-unknown-linux-${LIBC}"; exit; } ;; openrisc*:Linux:*:*) echo or1k-unknown-linux-${LIBC} exit ;; or32:Linux:*:* | or1k*:Linux:*:*) echo ${UNAME_MACHINE}-unknown-linux-${LIBC} exit ;; padre:Linux:*:*) echo sparc-unknown-linux-${LIBC} exit ;; parisc64:Linux:*:* | hppa64:Linux:*:*) echo hppa64-unknown-linux-${LIBC} exit ;; parisc:Linux:*:* | hppa:Linux:*:*) # Look for CPU level case `grep '^cpu[^a-z]*:' /proc/cpuinfo 2>/dev/null | cut -d' ' -f2` in PA7*) echo hppa1.1-unknown-linux-${LIBC} ;; PA8*) echo hppa2.0-unknown-linux-${LIBC} ;; *) echo hppa-unknown-linux-${LIBC} ;; esac exit ;; ppc64:Linux:*:*) echo powerpc64-unknown-linux-${LIBC} exit ;; ppc:Linux:*:*) echo powerpc-unknown-linux-${LIBC} exit ;; ppc64le:Linux:*:*) echo powerpc64le-unknown-linux-${LIBC} exit ;; ppcle:Linux:*:*) echo powerpcle-unknown-linux-${LIBC} exit ;; s390:Linux:*:* | s390x:Linux:*:*) echo ${UNAME_MACHINE}-ibm-linux-${LIBC} exit ;; sh64*:Linux:*:*) echo ${UNAME_MACHINE}-unknown-linux-${LIBC} exit ;; sh*:Linux:*:*) echo ${UNAME_MACHINE}-unknown-linux-${LIBC} exit ;; sparc:Linux:*:* | sparc64:Linux:*:*) echo ${UNAME_MACHINE}-unknown-linux-${LIBC} exit ;; tile*:Linux:*:*) echo ${UNAME_MACHINE}-unknown-linux-${LIBC} exit ;; vax:Linux:*:*) echo ${UNAME_MACHINE}-dec-linux-${LIBC} exit ;; x86_64:Linux:*:*) echo ${UNAME_MACHINE}-pc-linux-${LIBC} exit ;; xtensa*:Linux:*:*) echo ${UNAME_MACHINE}-unknown-linux-${LIBC} exit ;; i*86:DYNIX/ptx:4*:*) # ptx 4.0 does uname -s correctly, with DYNIX/ptx in there. # earlier versions are messed up and put the nodename in both # sysname and nodename. echo i386-sequent-sysv4 exit ;; i*86:UNIX_SV:4.2MP:2.*) # Unixware is an offshoot of SVR4, but it has its own version # number series starting with 2... # I am not positive that other SVR4 systems won't match this, # I just have to hope. -- rms. # Use sysv4.2uw... so that sysv4* matches it. echo ${UNAME_MACHINE}-pc-sysv4.2uw${UNAME_VERSION} exit ;; i*86:OS/2:*:*) # If we were able to find `uname', then EMX Unix compatibility # is probably installed. echo ${UNAME_MACHINE}-pc-os2-emx exit ;; i*86:XTS-300:*:STOP) echo ${UNAME_MACHINE}-unknown-stop exit ;; i*86:atheos:*:*) echo ${UNAME_MACHINE}-unknown-atheos exit ;; i*86:syllable:*:*) echo ${UNAME_MACHINE}-pc-syllable exit ;; i*86:LynxOS:2.*:* | i*86:LynxOS:3.[01]*:* | i*86:LynxOS:4.[02]*:*) echo i386-unknown-lynxos${UNAME_RELEASE} exit ;; i*86:*DOS:*:*) echo ${UNAME_MACHINE}-pc-msdosdjgpp exit ;; i*86:*:4.*:* | i*86:SYSTEM_V:4.*:*) UNAME_REL=`echo ${UNAME_RELEASE} | sed 's/\/MP$//'` if grep Novell /usr/include/link.h >/dev/null 2>/dev/null; then echo ${UNAME_MACHINE}-univel-sysv${UNAME_REL} else echo ${UNAME_MACHINE}-pc-sysv${UNAME_REL} fi exit ;; i*86:*:5:[678]*) # UnixWare 7.x, OpenUNIX and OpenServer 6. case `/bin/uname -X | grep "^Machine"` in *486*) UNAME_MACHINE=i486 ;; *Pentium) UNAME_MACHINE=i586 ;; *Pent*|*Celeron) UNAME_MACHINE=i686 ;; esac echo ${UNAME_MACHINE}-unknown-sysv${UNAME_RELEASE}${UNAME_SYSTEM}${UNAME_VERSION} exit ;; i*86:*:3.2:*) if test -f /usr/options/cb.name; then UNAME_REL=`sed -n 's/.*Version //p' /dev/null >/dev/null ; then UNAME_REL=`(/bin/uname -X|grep Release|sed -e 's/.*= //')` (/bin/uname -X|grep i80486 >/dev/null) && UNAME_MACHINE=i486 (/bin/uname -X|grep '^Machine.*Pentium' >/dev/null) \ && UNAME_MACHINE=i586 (/bin/uname -X|grep '^Machine.*Pent *II' >/dev/null) \ && UNAME_MACHINE=i686 (/bin/uname -X|grep '^Machine.*Pentium Pro' >/dev/null) \ && UNAME_MACHINE=i686 echo ${UNAME_MACHINE}-pc-sco$UNAME_REL else echo ${UNAME_MACHINE}-pc-sysv32 fi exit ;; pc:*:*:*) # Left here for compatibility: # uname -m prints for DJGPP always 'pc', but it prints nothing about # the processor, so we play safe by assuming i586. # Note: whatever this is, it MUST be the same as what config.sub # prints for the "djgpp" host, or else GDB configury will decide that # this is a cross-build. echo i586-pc-msdosdjgpp exit ;; Intel:Mach:3*:*) echo i386-pc-mach3 exit ;; paragon:*:*:*) echo i860-intel-osf1 exit ;; i860:*:4.*:*) # i860-SVR4 if grep Stardent /usr/include/sys/uadmin.h >/dev/null 2>&1 ; then echo i860-stardent-sysv${UNAME_RELEASE} # Stardent Vistra i860-SVR4 else # Add other i860-SVR4 vendors below as they are discovered. echo i860-unknown-sysv${UNAME_RELEASE} # Unknown i860-SVR4 fi exit ;; mini*:CTIX:SYS*5:*) # "miniframe" echo m68010-convergent-sysv exit ;; mc68k:UNIX:SYSTEM5:3.51m) echo m68k-convergent-sysv exit ;; M680?0:D-NIX:5.3:*) echo m68k-diab-dnix exit ;; M68*:*:R3V[5678]*:*) test -r /sysV68 && { echo 'm68k-motorola-sysv'; exit; } ;; 3[345]??:*:4.0:3.0 | 3[34]??A:*:4.0:3.0 | 3[34]??,*:*:4.0:3.0 | 3[34]??/*:*:4.0:3.0 | 4400:*:4.0:3.0 | 4850:*:4.0:3.0 | SKA40:*:4.0:3.0 | SDS2:*:4.0:3.0 | SHG2:*:4.0:3.0 | S7501*:*:4.0:3.0) OS_REL='' test -r /etc/.relid \ && OS_REL=.`sed -n 's/[^ ]* [^ ]* \([0-9][0-9]\).*/\1/p' < /etc/.relid` /bin/uname -p 2>/dev/null | grep 86 >/dev/null \ && { echo i486-ncr-sysv4.3${OS_REL}; exit; } /bin/uname -p 2>/dev/null | /bin/grep entium >/dev/null \ && { echo i586-ncr-sysv4.3${OS_REL}; exit; } ;; 3[34]??:*:4.0:* | 3[34]??,*:*:4.0:*) /bin/uname -p 2>/dev/null | grep 86 >/dev/null \ && { echo i486-ncr-sysv4; exit; } ;; NCR*:*:4.2:* | MPRAS*:*:4.2:*) OS_REL='.3' test -r /etc/.relid \ && OS_REL=.`sed -n 's/[^ ]* [^ ]* \([0-9][0-9]\).*/\1/p' < /etc/.relid` /bin/uname -p 2>/dev/null | grep 86 >/dev/null \ && { echo i486-ncr-sysv4.3${OS_REL}; exit; } /bin/uname -p 2>/dev/null | /bin/grep entium >/dev/null \ && { echo i586-ncr-sysv4.3${OS_REL}; exit; } /bin/uname -p 2>/dev/null | /bin/grep pteron >/dev/null \ && { echo i586-ncr-sysv4.3${OS_REL}; exit; } ;; m68*:LynxOS:2.*:* | m68*:LynxOS:3.0*:*) echo m68k-unknown-lynxos${UNAME_RELEASE} exit ;; mc68030:UNIX_System_V:4.*:*) echo m68k-atari-sysv4 exit ;; TSUNAMI:LynxOS:2.*:*) echo sparc-unknown-lynxos${UNAME_RELEASE} exit ;; rs6000:LynxOS:2.*:*) echo rs6000-unknown-lynxos${UNAME_RELEASE} exit ;; PowerPC:LynxOS:2.*:* | PowerPC:LynxOS:3.[01]*:* | PowerPC:LynxOS:4.[02]*:*) echo powerpc-unknown-lynxos${UNAME_RELEASE} exit ;; SM[BE]S:UNIX_SV:*:*) echo mips-dde-sysv${UNAME_RELEASE} exit ;; RM*:ReliantUNIX-*:*:*) echo mips-sni-sysv4 exit ;; RM*:SINIX-*:*:*) echo mips-sni-sysv4 exit ;; *:SINIX-*:*:*) if uname -p 2>/dev/null >/dev/null ; then UNAME_MACHINE=`(uname -p) 2>/dev/null` echo ${UNAME_MACHINE}-sni-sysv4 else echo ns32k-sni-sysv fi exit ;; PENTIUM:*:4.0*:*) # Unisys `ClearPath HMP IX 4000' SVR4/MP effort # says echo i586-unisys-sysv4 exit ;; *:UNIX_System_V:4*:FTX*) # From Gerald Hewes . # How about differentiating between stratus architectures? -djm echo hppa1.1-stratus-sysv4 exit ;; *:*:*:FTX*) # From seanf@swdc.stratus.com. echo i860-stratus-sysv4 exit ;; i*86:VOS:*:*) # From Paul.Green@stratus.com. echo ${UNAME_MACHINE}-stratus-vos exit ;; *:VOS:*:*) # From Paul.Green@stratus.com. echo hppa1.1-stratus-vos exit ;; mc68*:A/UX:*:*) echo m68k-apple-aux${UNAME_RELEASE} exit ;; news*:NEWS-OS:6*:*) echo mips-sony-newsos6 exit ;; R[34]000:*System_V*:*:* | R4000:UNIX_SYSV:*:* | R*000:UNIX_SV:*:*) if [ -d /usr/nec ]; then echo mips-nec-sysv${UNAME_RELEASE} else echo mips-unknown-sysv${UNAME_RELEASE} fi exit ;; BeBox:BeOS:*:*) # BeOS running on hardware made by Be, PPC only. echo powerpc-be-beos exit ;; BeMac:BeOS:*:*) # BeOS running on Mac or Mac clone, PPC only. echo powerpc-apple-beos exit ;; BePC:BeOS:*:*) # BeOS running on Intel PC compatible. echo i586-pc-beos exit ;; BePC:Haiku:*:*) # Haiku running on Intel PC compatible. echo i586-pc-haiku exit ;; x86_64:Haiku:*:*) echo x86_64-unknown-haiku exit ;; SX-4:SUPER-UX:*:*) echo sx4-nec-superux${UNAME_RELEASE} exit ;; SX-5:SUPER-UX:*:*) echo sx5-nec-superux${UNAME_RELEASE} exit ;; SX-6:SUPER-UX:*:*) echo sx6-nec-superux${UNAME_RELEASE} exit ;; SX-7:SUPER-UX:*:*) echo sx7-nec-superux${UNAME_RELEASE} exit ;; SX-8:SUPER-UX:*:*) echo sx8-nec-superux${UNAME_RELEASE} exit ;; SX-8R:SUPER-UX:*:*) echo sx8r-nec-superux${UNAME_RELEASE} exit ;; Power*:Rhapsody:*:*) echo powerpc-apple-rhapsody${UNAME_RELEASE} exit ;; *:Rhapsody:*:*) echo ${UNAME_MACHINE}-apple-rhapsody${UNAME_RELEASE} exit ;; *:Darwin:*:*) UNAME_PROCESSOR=`uname -p` || UNAME_PROCESSOR=unknown eval $set_cc_for_build if test "$UNAME_PROCESSOR" = unknown ; then UNAME_PROCESSOR=powerpc fi if test `echo "$UNAME_RELEASE" | sed -e 's/\..*//'` -le 10 ; then if [ "$CC_FOR_BUILD" != 'no_compiler_found' ]; then if (echo '#ifdef __LP64__'; echo IS_64BIT_ARCH; echo '#endif') | \ (CCOPTS= $CC_FOR_BUILD -E - 2>/dev/null) | \ grep IS_64BIT_ARCH >/dev/null then case $UNAME_PROCESSOR in i386) UNAME_PROCESSOR=x86_64 ;; powerpc) UNAME_PROCESSOR=powerpc64 ;; esac fi fi elif test "$UNAME_PROCESSOR" = i386 ; then # Avoid executing cc on OS X 10.9, as it ships with a stub # that puts up a graphical alert prompting to install # developer tools. Any system running Mac OS X 10.7 or # later (Darwin 11 and later) is required to have a 64-bit # processor. This is not true of the ARM version of Darwin # that Apple uses in portable devices. UNAME_PROCESSOR=x86_64 fi echo ${UNAME_PROCESSOR}-apple-darwin${UNAME_RELEASE} exit ;; *:procnto*:*:* | *:QNX:[0123456789]*:*) UNAME_PROCESSOR=`uname -p` if test "$UNAME_PROCESSOR" = "x86"; then UNAME_PROCESSOR=i386 UNAME_MACHINE=pc fi echo ${UNAME_PROCESSOR}-${UNAME_MACHINE}-nto-qnx${UNAME_RELEASE} exit ;; *:QNX:*:4*) echo i386-pc-qnx exit ;; NEO-?:NONSTOP_KERNEL:*:*) echo neo-tandem-nsk${UNAME_RELEASE} exit ;; NSE-*:NONSTOP_KERNEL:*:*) echo nse-tandem-nsk${UNAME_RELEASE} exit ;; NSR-?:NONSTOP_KERNEL:*:*) echo nsr-tandem-nsk${UNAME_RELEASE} exit ;; *:NonStop-UX:*:*) echo mips-compaq-nonstopux exit ;; BS2000:POSIX*:*:*) echo bs2000-siemens-sysv exit ;; DS/*:UNIX_System_V:*:*) echo ${UNAME_MACHINE}-${UNAME_SYSTEM}-${UNAME_RELEASE} exit ;; *:Plan9:*:*) # "uname -m" is not consistent, so use $cputype instead. 386 # is converted to i386 for consistency with other x86 # operating systems. if test "$cputype" = "386"; then UNAME_MACHINE=i386 else UNAME_MACHINE="$cputype" fi echo ${UNAME_MACHINE}-unknown-plan9 exit ;; *:TOPS-10:*:*) echo pdp10-unknown-tops10 exit ;; *:TENEX:*:*) echo pdp10-unknown-tenex exit ;; KS10:TOPS-20:*:* | KL10:TOPS-20:*:* | TYPE4:TOPS-20:*:*) echo pdp10-dec-tops20 exit ;; XKL-1:TOPS-20:*:* | TYPE5:TOPS-20:*:*) echo pdp10-xkl-tops20 exit ;; *:TOPS-20:*:*) echo pdp10-unknown-tops20 exit ;; *:ITS:*:*) echo pdp10-unknown-its exit ;; SEI:*:*:SEIUX) echo mips-sei-seiux${UNAME_RELEASE} exit ;; *:DragonFly:*:*) echo ${UNAME_MACHINE}-unknown-dragonfly`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'` exit ;; *:*VMS:*:*) UNAME_MACHINE=`(uname -p) 2>/dev/null` case "${UNAME_MACHINE}" in A*) echo alpha-dec-vms ; exit ;; I*) echo ia64-dec-vms ; exit ;; V*) echo vax-dec-vms ; exit ;; esac ;; *:XENIX:*:SysV) echo i386-pc-xenix exit ;; i*86:skyos:*:*) echo ${UNAME_MACHINE}-pc-skyos`echo ${UNAME_RELEASE}` | sed -e 's/ .*$//' exit ;; i*86:rdos:*:*) echo ${UNAME_MACHINE}-pc-rdos exit ;; i*86:AROS:*:*) echo ${UNAME_MACHINE}-pc-aros exit ;; x86_64:VMkernel:*:*) echo ${UNAME_MACHINE}-unknown-esx exit ;; esac cat >&2 < in order to provide the needed information to handle your system. config.guess timestamp = $timestamp uname -m = `(uname -m) 2>/dev/null || echo unknown` uname -r = `(uname -r) 2>/dev/null || echo unknown` uname -s = `(uname -s) 2>/dev/null || echo unknown` uname -v = `(uname -v) 2>/dev/null || echo unknown` /usr/bin/uname -p = `(/usr/bin/uname -p) 2>/dev/null` /bin/uname -X = `(/bin/uname -X) 2>/dev/null` hostinfo = `(hostinfo) 2>/dev/null` /bin/universe = `(/bin/universe) 2>/dev/null` /usr/bin/arch -k = `(/usr/bin/arch -k) 2>/dev/null` /bin/arch = `(/bin/arch) 2>/dev/null` /usr/bin/oslevel = `(/usr/bin/oslevel) 2>/dev/null` /usr/convex/getsysinfo = `(/usr/convex/getsysinfo) 2>/dev/null` UNAME_MACHINE = ${UNAME_MACHINE} UNAME_RELEASE = ${UNAME_RELEASE} UNAME_SYSTEM = ${UNAME_SYSTEM} UNAME_VERSION = ${UNAME_VERSION} EOF exit 1 # Local variables: # eval: (add-hook 'write-file-hooks 'time-stamp) # time-stamp-start: "timestamp='" # time-stamp-format: "%:y-%02m-%02d" # time-stamp-end: "'" # End: vowpal-wabbit-8.5.0.dfsg1/config.status000077500000000000000000002063651323713302100200310ustar00rootroot00000000000000#! /bin/bash # Generated by configure. # Run this file to recreate the current configuration. # Compiler output produced by configure, useful for debugging # configure, is in config.log if it exists. debug=false ac_cs_recheck=false ac_cs_silent=false SHELL=${CONFIG_SHELL-/bin/bash} export SHELL ## -------------------- ## ## M4sh Initialization. ## ## -------------------- ## # Be more Bourne compatible DUALCASE=1; export DUALCASE # for MKS sh if test -n "${ZSH_VERSION+set}" && (emulate sh) >/dev/null 2>&1; then : emulate sh NULLCMD=: # Pre-4.2 versions of Zsh do word splitting on ${1+"$@"}, which # is contrary to our usage. Disable this feature. alias -g '${1+"$@"}'='"$@"' setopt NO_GLOB_SUBST else case `(set -o) 2>/dev/null` in #( *posix*) : set -o posix ;; #( *) : ;; esac fi as_nl=' ' export as_nl # Printing a long string crashes Solaris 7 /usr/bin/printf. as_echo='\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\' as_echo=$as_echo$as_echo$as_echo$as_echo$as_echo as_echo=$as_echo$as_echo$as_echo$as_echo$as_echo$as_echo # Prefer a ksh shell builtin over an external printf program on Solaris, # but without wasting forks for bash or zsh. if test -z "$BASH_VERSION$ZSH_VERSION" \ && (test "X`print -r -- $as_echo`" = "X$as_echo") 2>/dev/null; then as_echo='print -r --' as_echo_n='print -rn --' elif (test "X`printf %s $as_echo`" = "X$as_echo") 2>/dev/null; then as_echo='printf %s\n' as_echo_n='printf %s' else if test "X`(/usr/ucb/echo -n -n $as_echo) 2>/dev/null`" = "X-n $as_echo"; then as_echo_body='eval /usr/ucb/echo -n "$1$as_nl"' as_echo_n='/usr/ucb/echo -n' else as_echo_body='eval expr "X$1" : "X\\(.*\\)"' as_echo_n_body='eval arg=$1; case $arg in #( *"$as_nl"*) expr "X$arg" : "X\\(.*\\)$as_nl"; arg=`expr "X$arg" : ".*$as_nl\\(.*\\)"`;; esac; expr "X$arg" : "X\\(.*\\)" | tr -d "$as_nl" ' export as_echo_n_body as_echo_n='sh -c $as_echo_n_body as_echo' fi export as_echo_body as_echo='sh -c $as_echo_body as_echo' fi # The user is always right. if test "${PATH_SEPARATOR+set}" != set; then PATH_SEPARATOR=: (PATH='/bin;/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 && { (PATH='/bin:/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 || PATH_SEPARATOR=';' } fi # IFS # We need space, tab and new line, in precisely that order. Quoting is # there to prevent editors from complaining about space-tab. # (If _AS_PATH_WALK were called with IFS unset, it would disable word # splitting by setting IFS to empty value.) IFS=" "" $as_nl" # Find who we are. Look in the path if we contain no directory separator. as_myself= case $0 in #(( *[\\/]* ) as_myself=$0 ;; *) as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. test -r "$as_dir/$0" && as_myself=$as_dir/$0 && break done IFS=$as_save_IFS ;; esac # We did not find ourselves, most probably we were run as `sh COMMAND' # in which case we are not to be found in the path. if test "x$as_myself" = x; then as_myself=$0 fi if test ! -f "$as_myself"; then $as_echo "$as_myself: error: cannot find myself; rerun with an absolute file name" >&2 exit 1 fi # Unset variables that we do not need and which cause bugs (e.g. in # pre-3.0 UWIN ksh). But do not cause bugs in bash 2.01; the "|| exit 1" # suppresses any "Segmentation fault" message there. '((' could # trigger a bug in pdksh 5.2.14. for as_var in BASH_ENV ENV MAIL MAILPATH do eval test x\${$as_var+set} = xset \ && ( (unset $as_var) || exit 1) >/dev/null 2>&1 && unset $as_var || : done PS1='$ ' PS2='> ' PS4='+ ' # NLS nuisances. LC_ALL=C export LC_ALL LANGUAGE=C export LANGUAGE # CDPATH. (unset CDPATH) >/dev/null 2>&1 && unset CDPATH # as_fn_error STATUS ERROR [LINENO LOG_FD] # ---------------------------------------- # Output "`basename $0`: error: ERROR" to stderr. If LINENO and LOG_FD are # provided, also output the error to LOG_FD, referencing LINENO. Then exit the # script with STATUS, using 1 if that was 0. as_fn_error () { as_status=$1; test $as_status -eq 0 && as_status=1 if test "$4"; then as_lineno=${as_lineno-"$3"} as_lineno_stack=as_lineno_stack=$as_lineno_stack $as_echo "$as_me:${as_lineno-$LINENO}: error: $2" >&$4 fi $as_echo "$as_me: error: $2" >&2 as_fn_exit $as_status } # as_fn_error # as_fn_set_status STATUS # ----------------------- # Set $? to STATUS, without forking. as_fn_set_status () { return $1 } # as_fn_set_status # as_fn_exit STATUS # ----------------- # Exit the shell with STATUS, even in a "trap 0" or "set -e" context. as_fn_exit () { set +e as_fn_set_status $1 exit $1 } # as_fn_exit # as_fn_unset VAR # --------------- # Portably unset VAR. as_fn_unset () { { eval $1=; unset $1;} } as_unset=as_fn_unset # as_fn_append VAR VALUE # ---------------------- # Append the text in VALUE to the end of the definition contained in VAR. Take # advantage of any shell optimizations that allow amortized linear growth over # repeated appends, instead of the typical quadratic growth present in naive # implementations. if (eval "as_var=1; as_var+=2; test x\$as_var = x12") 2>/dev/null; then : eval 'as_fn_append () { eval $1+=\$2 }' else as_fn_append () { eval $1=\$$1\$2 } fi # as_fn_append # as_fn_arith ARG... # ------------------ # Perform arithmetic evaluation on the ARGs, and store the result in the # global $as_val. Take advantage of shells that can avoid forks. The arguments # must be portable across $(()) and expr. if (eval "test \$(( 1 + 1 )) = 2") 2>/dev/null; then : eval 'as_fn_arith () { as_val=$(( $* )) }' else as_fn_arith () { as_val=`expr "$@" || test $? -eq 1` } fi # as_fn_arith if expr a : '\(a\)' >/dev/null 2>&1 && test "X`expr 00001 : '.*\(...\)'`" = X001; then as_expr=expr else as_expr=false fi if (basename -- /) >/dev/null 2>&1 && test "X`basename -- / 2>&1`" = "X/"; then as_basename=basename else as_basename=false fi if (as_dir=`dirname -- /` && test "X$as_dir" = X/) >/dev/null 2>&1; then as_dirname=dirname else as_dirname=false fi as_me=`$as_basename -- "$0" || $as_expr X/"$0" : '.*/\([^/][^/]*\)/*$' \| \ X"$0" : 'X\(//\)$' \| \ X"$0" : 'X\(/\)' \| . 2>/dev/null || $as_echo X/"$0" | sed '/^.*\/\([^/][^/]*\)\/*$/{ s//\1/ q } /^X\/\(\/\/\)$/{ s//\1/ q } /^X\/\(\/\).*/{ s//\1/ q } s/.*/./; q'` # Avoid depending upon Character Ranges. as_cr_letters='abcdefghijklmnopqrstuvwxyz' as_cr_LETTERS='ABCDEFGHIJKLMNOPQRSTUVWXYZ' as_cr_Letters=$as_cr_letters$as_cr_LETTERS as_cr_digits='0123456789' as_cr_alnum=$as_cr_Letters$as_cr_digits ECHO_C= ECHO_N= ECHO_T= case `echo -n x` in #((((( -n*) case `echo 'xy\c'` in *c*) ECHO_T=' ';; # ECHO_T is single tab character. xy) ECHO_C='\c';; *) echo `echo ksh88 bug on AIX 6.1` > /dev/null ECHO_T=' ';; esac;; *) ECHO_N='-n';; esac rm -f conf$$ conf$$.exe conf$$.file if test -d conf$$.dir; then rm -f conf$$.dir/conf$$.file else rm -f conf$$.dir mkdir conf$$.dir 2>/dev/null fi if (echo >conf$$.file) 2>/dev/null; then if ln -s conf$$.file conf$$ 2>/dev/null; then as_ln_s='ln -s' # ... but there are two gotchas: # 1) On MSYS, both `ln -s file dir' and `ln file dir' fail. # 2) DJGPP < 2.04 has no symlinks; `ln -s' creates a wrapper executable. # In both cases, we have to default to `cp -pR'. ln -s conf$$.file conf$$.dir 2>/dev/null && test ! -f conf$$.exe || as_ln_s='cp -pR' elif ln conf$$.file conf$$ 2>/dev/null; then as_ln_s=ln else as_ln_s='cp -pR' fi else as_ln_s='cp -pR' fi rm -f conf$$ conf$$.exe conf$$.dir/conf$$.file conf$$.file rmdir conf$$.dir 2>/dev/null # as_fn_mkdir_p # ------------- # Create "$as_dir" as a directory, including parents if necessary. as_fn_mkdir_p () { case $as_dir in #( -*) as_dir=./$as_dir;; esac test -d "$as_dir" || eval $as_mkdir_p || { as_dirs= while :; do case $as_dir in #( *\'*) as_qdir=`$as_echo "$as_dir" | sed "s/'/'\\\\\\\\''/g"`;; #'( *) as_qdir=$as_dir;; esac as_dirs="'$as_qdir' $as_dirs" as_dir=`$as_dirname -- "$as_dir" || $as_expr X"$as_dir" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ X"$as_dir" : 'X\(//\)[^/]' \| \ X"$as_dir" : 'X\(//\)$' \| \ X"$as_dir" : 'X\(/\)' \| . 2>/dev/null || $as_echo X"$as_dir" | sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ s//\1/ q } /^X\(\/\/\)[^/].*/{ s//\1/ q } /^X\(\/\/\)$/{ s//\1/ q } /^X\(\/\).*/{ s//\1/ q } s/.*/./; q'` test -d "$as_dir" && break done test -z "$as_dirs" || eval "mkdir $as_dirs" } || test -d "$as_dir" || as_fn_error $? "cannot create directory $as_dir" } # as_fn_mkdir_p if mkdir -p . 2>/dev/null; then as_mkdir_p='mkdir -p "$as_dir"' else test -d ./-p && rmdir ./-p as_mkdir_p=false fi # as_fn_executable_p FILE # ----------------------- # Test if FILE is an executable regular file. as_fn_executable_p () { test -f "$1" && test -x "$1" } # as_fn_executable_p as_test_x='test -x' as_executable_p=as_fn_executable_p # Sed expression to map a string onto a valid CPP name. as_tr_cpp="eval sed 'y%*$as_cr_letters%P$as_cr_LETTERS%;s%[^_$as_cr_alnum]%_%g'" # Sed expression to map a string onto a valid variable name. as_tr_sh="eval sed 'y%*+%pp%;s%[^_$as_cr_alnum]%_%g'" exec 6>&1 ## ----------------------------------- ## ## Main body of $CONFIG_STATUS script. ## ## ----------------------------------- ## # Save the log message, to keep $0 and so on meaningful, and to # report actual input values of CONFIG_FILES etc. instead of their # values after options handling. ac_log=" This file was extended by vowpal_wabbit $as_me 8.5.0, which was generated by GNU Autoconf 2.69. Invocation command line was CONFIG_FILES = $CONFIG_FILES CONFIG_HEADERS = $CONFIG_HEADERS CONFIG_LINKS = $CONFIG_LINKS CONFIG_COMMANDS = $CONFIG_COMMANDS $ $0 $@ on `(hostname || uname -n) 2>/dev/null | sed 1q` " # Files that config.status was made for. config_files=" Makefile vowpalwabbit/Makefile cluster/Makefile library/Makefile libvw.pc libvw_c_wrapper.pc" config_headers=" vowpalwabbit/config.h" config_commands=" depfiles libtool" ac_cs_usage="\ \`$as_me' instantiates files and other configuration actions from templates according to the current configuration. Unless the files and actions are specified as TAGs, all are instantiated by default. Usage: $0 [OPTION]... [TAG]... -h, --help print this help, then exit -V, --version print version number and configuration settings, then exit --config print configuration, then exit -q, --quiet, --silent do not print progress messages -d, --debug don't remove temporary files --recheck update $as_me by reconfiguring in the same conditions --file=FILE[:TEMPLATE] instantiate the configuration file FILE --header=FILE[:TEMPLATE] instantiate the configuration header FILE Configuration files: $config_files Configuration headers: $config_headers Configuration commands: $config_commands Report bugs to . vowpal_wabbit home page: ." ac_cs_config="'--with-boost-libdir=/usr/lib/x86_64-linux-gnu' 'CXX=g++'" ac_cs_version="\ vowpal_wabbit config.status 8.5.0 configured by ./configure, generated by GNU Autoconf 2.69, with options \"$ac_cs_config\" Copyright (C) 2012 Free Software Foundation, Inc. This config.status script is free software; the Free Software Foundation gives unlimited permission to copy, distribute and modify it." ac_pwd='/home/jcl/programs/vowpal_wabbit' srcdir='.' INSTALL='/usr/bin/install -c' MKDIR_P='/bin/mkdir -p' AWK='gawk' test -n "$AWK" || AWK=awk # The default lists apply if the user does not specify any file. ac_need_defaults=: while test $# != 0 do case $1 in --*=?*) ac_option=`expr "X$1" : 'X\([^=]*\)='` ac_optarg=`expr "X$1" : 'X[^=]*=\(.*\)'` ac_shift=: ;; --*=) ac_option=`expr "X$1" : 'X\([^=]*\)='` ac_optarg= ac_shift=: ;; *) ac_option=$1 ac_optarg=$2 ac_shift=shift ;; esac case $ac_option in # Handling of the options. -recheck | --recheck | --rechec | --reche | --rech | --rec | --re | --r) ac_cs_recheck=: ;; --version | --versio | --versi | --vers | --ver | --ve | --v | -V ) $as_echo "$ac_cs_version"; exit ;; --config | --confi | --conf | --con | --co | --c ) $as_echo "$ac_cs_config"; exit ;; --debug | --debu | --deb | --de | --d | -d ) debug=: ;; --file | --fil | --fi | --f ) $ac_shift case $ac_optarg in *\'*) ac_optarg=`$as_echo "$ac_optarg" | sed "s/'/'\\\\\\\\''/g"` ;; '') as_fn_error $? "missing file argument" ;; esac as_fn_append CONFIG_FILES " '$ac_optarg'" ac_need_defaults=false;; --header | --heade | --head | --hea ) $ac_shift case $ac_optarg in *\'*) ac_optarg=`$as_echo "$ac_optarg" | sed "s/'/'\\\\\\\\''/g"` ;; esac as_fn_append CONFIG_HEADERS " '$ac_optarg'" ac_need_defaults=false;; --he | --h) # Conflict between --help and --header as_fn_error $? "ambiguous option: \`$1' Try \`$0 --help' for more information.";; --help | --hel | -h ) $as_echo "$ac_cs_usage"; exit ;; -q | -quiet | --quiet | --quie | --qui | --qu | --q \ | -silent | --silent | --silen | --sile | --sil | --si | --s) ac_cs_silent=: ;; # This is an error. -*) as_fn_error $? "unrecognized option: \`$1' Try \`$0 --help' for more information." ;; *) as_fn_append ac_config_targets " $1" ac_need_defaults=false ;; esac shift done ac_configure_extra_args= if $ac_cs_silent; then exec 6>/dev/null ac_configure_extra_args="$ac_configure_extra_args --silent" fi if $ac_cs_recheck; then set X /bin/bash './configure' '--with-boost-libdir=/usr/lib/x86_64-linux-gnu' 'CXX=g++' $ac_configure_extra_args --no-create --no-recursion shift $as_echo "running CONFIG_SHELL=/bin/bash $*" >&6 CONFIG_SHELL='/bin/bash' export CONFIG_SHELL exec "$@" fi exec 5>>config.log { echo sed 'h;s/./-/g;s/^.../## /;s/...$/ ##/;p;x;p;x' <<_ASBOX ## Running $as_me. ## _ASBOX $as_echo "$ac_log" } >&5 # # INIT-COMMANDS # AMDEP_TRUE="" ac_aux_dir="." # The HP-UX ksh and POSIX shell print the target directory to stdout # if CDPATH is set. (unset CDPATH) >/dev/null 2>&1 && unset CDPATH sed_quote_subst='s/\(["`$\\]\)/\\\1/g' double_quote_subst='s/\(["`\\]\)/\\\1/g' delay_variable_subst='s/\\\\\\\\\\\$/\\\\\\$/g' macro_version='2.4.6' macro_revision='2.4.6' enable_shared='yes' enable_static='yes' pic_mode='default' enable_fast_install='yes' shared_archive_member_spec='' SHELL='/bin/bash' ECHO='printf %s\n' PATH_SEPARATOR=':' host_alias='' host='x86_64-pc-linux-gnu' host_os='linux-gnu' build_alias='' build='x86_64-pc-linux-gnu' build_os='linux-gnu' SED='/bin/sed' Xsed='/bin/sed -e 1s/^X//' GREP='/bin/grep' EGREP='/bin/grep -E' FGREP='/bin/grep -F' LD='/usr/bin/ld -m elf_x86_64' NM='/usr/bin/nm -B' LN_S='ln -s' max_cmd_len='1572864' ac_objext='o' exeext='' lt_unset='unset' lt_SP2NL='tr \040 \012' lt_NL2SP='tr \015\012 \040\040' lt_cv_to_host_file_cmd='func_convert_file_noop' lt_cv_to_tool_file_cmd='func_convert_file_noop' reload_flag=' -r' reload_cmds='$LD$reload_flag -o $output$reload_objs' OBJDUMP='objdump' deplibs_check_method='pass_all' file_magic_cmd='$MAGIC_CMD' file_magic_glob='' want_nocaseglob='no' DLLTOOL='false' sharedlib_from_linklib_cmd='printf %s\n' AR='ar' AR_FLAGS='cru' archiver_list_spec='@' STRIP='strip' RANLIB='ranlib' old_postinstall_cmds='chmod 644 $oldlib~$RANLIB $tool_oldlib' old_postuninstall_cmds='' old_archive_cmds='$AR $AR_FLAGS $oldlib$oldobjs~$RANLIB $tool_oldlib' lock_old_archive_extraction='no' CC='gcc' CFLAGS='-g -O2' compiler='g++' GCC='yes' lt_cv_sys_global_symbol_pipe='sed -n -e '\''s/^.*[ ]\([ABCDGIRSTW][ABCDGIRSTW]*\)[ ][ ]*\([_A-Za-z][_A-Za-z0-9]*\)$/\1 \2 \2/p'\'' | sed '\''/ __gnu_lto/d'\''' lt_cv_sys_global_symbol_to_cdecl='sed -n -e '\''s/^T .* \(.*\)$/extern int \1();/p'\'' -e '\''s/^[ABCDGIRSTW][ABCDGIRSTW]* .* \(.*\)$/extern char \1;/p'\''' lt_cv_sys_global_symbol_to_import='' lt_cv_sys_global_symbol_to_c_name_address='sed -n -e '\''s/^: \(.*\) .*$/ {"\1", (void *) 0},/p'\'' -e '\''s/^[ABCDGIRSTW][ABCDGIRSTW]* .* \(.*\)$/ {"\1", (void *) \&\1},/p'\''' lt_cv_sys_global_symbol_to_c_name_address_lib_prefix='sed -n -e '\''s/^: \(.*\) .*$/ {"\1", (void *) 0},/p'\'' -e '\''s/^[ABCDGIRSTW][ABCDGIRSTW]* .* \(lib.*\)$/ {"\1", (void *) \&\1},/p'\'' -e '\''s/^[ABCDGIRSTW][ABCDGIRSTW]* .* \(.*\)$/ {"lib\1", (void *) \&\1},/p'\''' lt_cv_nm_interface='BSD nm' nm_file_list_spec='@' lt_sysroot='' lt_cv_truncate_bin='/bin/dd bs=4096 count=1' objdir='.libs' MAGIC_CMD='file' lt_prog_compiler_no_builtin_flag=' -fno-builtin' lt_prog_compiler_pic=' -fPIC -DPIC' lt_prog_compiler_wl='-Wl,' lt_prog_compiler_static='-static' lt_cv_prog_compiler_c_o='yes' need_locks='no' MANIFEST_TOOL=':' DSYMUTIL='' NMEDIT='' LIPO='' OTOOL='' OTOOL64='' libext='a' shrext_cmds='.so' extract_expsyms_cmds='' archive_cmds_need_lc='no' enable_shared_with_static_runtimes='no' export_dynamic_flag_spec='$wl--export-dynamic' whole_archive_flag_spec='$wl--whole-archive$convenience $wl--no-whole-archive' compiler_needs_object='no' old_archive_from_new_cmds='' old_archive_from_expsyms_cmds='' archive_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' archive_expsym_cmds='echo "{ global:" > $output_objdir/$libname.ver~ cat $export_symbols | sed -e "s/\(.*\)/\1;/" >> $output_objdir/$libname.ver~ echo "local: *; };" >> $output_objdir/$libname.ver~ $CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-version-script $wl$output_objdir/$libname.ver -o $lib' module_cmds='' module_expsym_cmds='' with_gnu_ld='yes' allow_undefined_flag='' no_undefined_flag='' hardcode_libdir_flag_spec='$wl-rpath $wl$libdir' hardcode_libdir_separator='' hardcode_direct='no' hardcode_direct_absolute='no' hardcode_minus_L='no' hardcode_shlibpath_var='unsupported' hardcode_automatic='no' inherit_rpath='no' link_all_deplibs='no' always_export_symbols='no' export_symbols_cmds='$NM $libobjs $convenience | $global_symbol_pipe | $SED '\''s/.* //'\'' | sort | uniq > $export_symbols' exclude_expsyms='_GLOBAL_OFFSET_TABLE_|_GLOBAL__F[ID]_.*' include_expsyms='' prelink_cmds='' postlink_cmds='' file_list_spec='' variables_saved_for_relink='PATH LD_LIBRARY_PATH LD_RUN_PATH GCC_EXEC_PREFIX COMPILER_PATH LIBRARY_PATH' need_lib_prefix='no' need_version='no' version_type='linux' runpath_var='LD_RUN_PATH' shlibpath_var='LD_LIBRARY_PATH' shlibpath_overrides_runpath='no' libname_spec='lib$name' library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' install_override_mode='' postinstall_cmds='' postuninstall_cmds='' finish_cmds='PATH="\$PATH:/sbin" ldconfig -n $libdir' finish_eval='' hardcode_into_libs='yes' sys_lib_search_path_spec='/usr/lib/gcc/x86_64-linux-gnu/5 /usr/lib/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib ' configure_time_dlsearch_path='/lib /usr/lib /usr/lib/x86_64-linux-gnu/libfakeroot /lib/i386-linux-gnu /usr/lib/i386-linux-gnu /lib/i686-linux-gnu /usr/lib/i686-linux-gnu /usr/lib/i386-linux-gnu/mesa /usr/local/lib /lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/mesa-egl /usr/lib/x86_64-linux-gnu/mesa ' configure_time_lt_sys_library_path='' hardcode_action='immediate' enable_dlopen='unknown' enable_dlopen_self='unknown' enable_dlopen_self_static='unknown' old_striplib='strip --strip-debug' striplib='strip --strip-unneeded' compiler_lib_search_dirs='' predep_objects='' postdep_objects='' predeps='' postdeps='' compiler_lib_search_path='' LD_CXX='/usr/bin/ld -m elf_x86_64' reload_flag_CXX=' -r' reload_cmds_CXX='$LD$reload_flag -o $output$reload_objs' old_archive_cmds_CXX='$AR $AR_FLAGS $oldlib$oldobjs~$RANLIB $tool_oldlib' compiler_CXX='g++' GCC_CXX='yes' lt_prog_compiler_no_builtin_flag_CXX=' -fno-builtin' lt_prog_compiler_pic_CXX=' -fPIC -DPIC' lt_prog_compiler_wl_CXX='-Wl,' lt_prog_compiler_static_CXX='-static' lt_cv_prog_compiler_c_o_CXX='yes' archive_cmds_need_lc_CXX='no' enable_shared_with_static_runtimes_CXX='no' export_dynamic_flag_spec_CXX='$wl--export-dynamic' whole_archive_flag_spec_CXX='$wl--whole-archive$convenience $wl--no-whole-archive' compiler_needs_object_CXX='no' old_archive_from_new_cmds_CXX='' old_archive_from_expsyms_cmds_CXX='' archive_cmds_CXX='$CC $pic_flag -shared -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname -o $lib' archive_expsym_cmds_CXX='$CC $pic_flag -shared -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' module_cmds_CXX='' module_expsym_cmds_CXX='' with_gnu_ld_CXX='yes' allow_undefined_flag_CXX='' no_undefined_flag_CXX='' hardcode_libdir_flag_spec_CXX='$wl-rpath $wl$libdir' hardcode_libdir_separator_CXX='' hardcode_direct_CXX='no' hardcode_direct_absolute_CXX='no' hardcode_minus_L_CXX='no' hardcode_shlibpath_var_CXX='unsupported' hardcode_automatic_CXX='no' inherit_rpath_CXX='no' link_all_deplibs_CXX='no' always_export_symbols_CXX='no' export_symbols_cmds_CXX='$NM $libobjs $convenience | $global_symbol_pipe | $SED '\''s/.* //'\'' | sort | uniq > $export_symbols' exclude_expsyms_CXX='_GLOBAL_OFFSET_TABLE_|_GLOBAL__F[ID]_.*' include_expsyms_CXX='' prelink_cmds_CXX='' postlink_cmds_CXX='' file_list_spec_CXX='' hardcode_action_CXX='immediate' compiler_lib_search_dirs_CXX='/usr/lib/gcc/x86_64-linux-gnu/5 /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu /usr/lib/gcc/x86_64-linux-gnu/5/../../../../lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /usr/lib/gcc/x86_64-linux-gnu/5/../../..' predep_objects_CXX='/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/5/crtbeginS.o' postdep_objects_CXX='/usr/lib/gcc/x86_64-linux-gnu/5/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crtn.o' predeps_CXX='' postdeps_CXX='-lstdc++ -lm -lgcc_s -lc -lgcc_s' compiler_lib_search_path_CXX='-L/usr/lib/gcc/x86_64-linux-gnu/5 -L/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/5/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/5/../../..' LTCC='gcc' LTCFLAGS='-g -O2' compiler='gcc' # A function that is used when there is no print builtin or printf. func_fallback_echo () { eval 'cat <<_LTECHO_EOF $1 _LTECHO_EOF' } # Quote evaled strings. for var in SHELL ECHO PATH_SEPARATOR SED GREP EGREP FGREP LD NM LN_S lt_SP2NL lt_NL2SP reload_flag OBJDUMP deplibs_check_method file_magic_cmd file_magic_glob want_nocaseglob DLLTOOL sharedlib_from_linklib_cmd AR AR_FLAGS archiver_list_spec STRIP RANLIB CC CFLAGS compiler lt_cv_sys_global_symbol_pipe lt_cv_sys_global_symbol_to_cdecl lt_cv_sys_global_symbol_to_import lt_cv_sys_global_symbol_to_c_name_address lt_cv_sys_global_symbol_to_c_name_address_lib_prefix lt_cv_nm_interface nm_file_list_spec lt_cv_truncate_bin lt_prog_compiler_no_builtin_flag lt_prog_compiler_pic lt_prog_compiler_wl lt_prog_compiler_static lt_cv_prog_compiler_c_o need_locks MANIFEST_TOOL DSYMUTIL NMEDIT LIPO OTOOL OTOOL64 shrext_cmds export_dynamic_flag_spec whole_archive_flag_spec compiler_needs_object with_gnu_ld allow_undefined_flag no_undefined_flag hardcode_libdir_flag_spec hardcode_libdir_separator exclude_expsyms include_expsyms file_list_spec variables_saved_for_relink libname_spec library_names_spec soname_spec install_override_mode finish_eval old_striplib striplib compiler_lib_search_dirs predep_objects postdep_objects predeps postdeps compiler_lib_search_path LD_CXX reload_flag_CXX compiler_CXX lt_prog_compiler_no_builtin_flag_CXX lt_prog_compiler_pic_CXX lt_prog_compiler_wl_CXX lt_prog_compiler_static_CXX lt_cv_prog_compiler_c_o_CXX export_dynamic_flag_spec_CXX whole_archive_flag_spec_CXX compiler_needs_object_CXX with_gnu_ld_CXX allow_undefined_flag_CXX no_undefined_flag_CXX hardcode_libdir_flag_spec_CXX hardcode_libdir_separator_CXX exclude_expsyms_CXX include_expsyms_CXX file_list_spec_CXX compiler_lib_search_dirs_CXX predep_objects_CXX postdep_objects_CXX predeps_CXX postdeps_CXX compiler_lib_search_path_CXX; do case `eval \\$ECHO \\""\\$$var"\\"` in *[\\\`\"\$]*) eval "lt_$var=\\\"\`\$ECHO \"\$$var\" | \$SED \"\$sed_quote_subst\"\`\\\"" ## exclude from sc_prohibit_nested_quotes ;; *) eval "lt_$var=\\\"\$$var\\\"" ;; esac done # Double-quote double-evaled strings. for var in reload_cmds old_postinstall_cmds old_postuninstall_cmds old_archive_cmds extract_expsyms_cmds old_archive_from_new_cmds old_archive_from_expsyms_cmds archive_cmds archive_expsym_cmds module_cmds module_expsym_cmds export_symbols_cmds prelink_cmds postlink_cmds postinstall_cmds postuninstall_cmds finish_cmds sys_lib_search_path_spec configure_time_dlsearch_path configure_time_lt_sys_library_path reload_cmds_CXX old_archive_cmds_CXX old_archive_from_new_cmds_CXX old_archive_from_expsyms_cmds_CXX archive_cmds_CXX archive_expsym_cmds_CXX module_cmds_CXX module_expsym_cmds_CXX export_symbols_cmds_CXX prelink_cmds_CXX postlink_cmds_CXX; do case `eval \\$ECHO \\""\\$$var"\\"` in *[\\\`\"\$]*) eval "lt_$var=\\\"\`\$ECHO \"\$$var\" | \$SED -e \"\$double_quote_subst\" -e \"\$sed_quote_subst\" -e \"\$delay_variable_subst\"\`\\\"" ## exclude from sc_prohibit_nested_quotes ;; *) eval "lt_$var=\\\"\$$var\\\"" ;; esac done ac_aux_dir='.' # See if we are running on zsh, and set the options that allow our # commands through without removal of \ escapes INIT. if test -n "${ZSH_VERSION+set}"; then setopt NO_GLOB_SUBST fi PACKAGE='vowpal_wabbit' VERSION='8.5.0' RM='rm -f' ofile='libtool' # Handling of arguments. for ac_config_target in $ac_config_targets do case $ac_config_target in "vowpalwabbit/config.h") CONFIG_HEADERS="$CONFIG_HEADERS vowpalwabbit/config.h" ;; "depfiles") CONFIG_COMMANDS="$CONFIG_COMMANDS depfiles" ;; "libtool") CONFIG_COMMANDS="$CONFIG_COMMANDS libtool" ;; "Makefile") CONFIG_FILES="$CONFIG_FILES Makefile" ;; "vowpalwabbit/Makefile") CONFIG_FILES="$CONFIG_FILES vowpalwabbit/Makefile" ;; "cluster/Makefile") CONFIG_FILES="$CONFIG_FILES cluster/Makefile" ;; "library/Makefile") CONFIG_FILES="$CONFIG_FILES library/Makefile" ;; "libvw.pc") CONFIG_FILES="$CONFIG_FILES libvw.pc" ;; "libvw_c_wrapper.pc") CONFIG_FILES="$CONFIG_FILES libvw_c_wrapper.pc" ;; *) as_fn_error $? "invalid argument: \`$ac_config_target'" "$LINENO" 5;; esac done # If the user did not use the arguments to specify the items to instantiate, # then the envvar interface is used. Set only those that are not. # We use the long form for the default assignment because of an extremely # bizarre bug on SunOS 4.1.3. if $ac_need_defaults; then test "${CONFIG_FILES+set}" = set || CONFIG_FILES=$config_files test "${CONFIG_HEADERS+set}" = set || CONFIG_HEADERS=$config_headers test "${CONFIG_COMMANDS+set}" = set || CONFIG_COMMANDS=$config_commands fi # Have a temporary directory for convenience. Make it in the build tree # simply because there is no reason against having it here, and in addition, # creating and moving files from /tmp can sometimes cause problems. # Hook for its removal unless debugging. # Note that there is a small window in which the directory will not be cleaned: # after its creation but before its name has been assigned to `$tmp'. $debug || { tmp= ac_tmp= trap 'exit_status=$? : "${ac_tmp:=$tmp}" { test ! -d "$ac_tmp" || rm -fr "$ac_tmp"; } && exit $exit_status ' 0 trap 'as_fn_exit 1' 1 2 13 15 } # Create a (secure) tmp directory for tmp files. { tmp=`(umask 077 && mktemp -d "./confXXXXXX") 2>/dev/null` && test -d "$tmp" } || { tmp=./conf$$-$RANDOM (umask 077 && mkdir "$tmp") } || as_fn_error $? "cannot create a temporary directory in ." "$LINENO" 5 ac_tmp=$tmp # Set up the scripts for CONFIG_FILES section. # No need to generate them if there are no CONFIG_FILES. # This happens for instance with `./config.status config.h'. if test -n "$CONFIG_FILES"; then ac_cr=`echo X | tr X '\015'` # On cygwin, bash can eat \r inside `` if the user requested igncr. # But we know of no other shell where ac_cr would be empty at this # point, so we can use a bashism as a fallback. if test "x$ac_cr" = x; then eval ac_cr=\$\'\\r\' fi ac_cs_awk_cr=`$AWK 'BEGIN { print "a\rb" }' /dev/null` if test "$ac_cs_awk_cr" = "a${ac_cr}b"; then ac_cs_awk_cr='\\r' else ac_cs_awk_cr=$ac_cr fi echo 'BEGIN {' >"$ac_tmp/subs1.awk" && cat >>"$ac_tmp/subs1.awk" <<\_ACAWK && S["am__EXEEXT_FALSE"]="" S["am__EXEEXT_TRUE"]="#" S["LTLIBOBJS"]="" S["LIBOBJS"]="" S["FREEBSD_FALSE"]="" S["FREEBSD_TRUE"]="#" S["CLANG_LIBCXX_FALSE"]="" S["CLANG_LIBCXX_TRUE"]="#" S["VWBUG_FALSE"]="" S["VWBUG_TRUE"]="#" S["PROFILE_FALSE"]="" S["PROFILE_TRUE"]="#" S["PARALLELIZE_FALSE"]="" S["PARALLELIZE_TRUE"]="#" S["NITPICK_FALSE"]="" S["NITPICK_TRUE"]="#" S["PTHREAD_CFLAGS"]="" S["PTHREAD_LIBS"]="-lpthread" S["PTHREAD_CC"]="gcc" S["ax_pthread_config"]="" S["ZLIB_LDFLAGS"]="-L/lib -lz" S["ZLIB_CPPFLAGS"]="-I/include" S["BOOST_PROGRAM_OPTIONS_LIB"]="-lboost_program_options" S["BOOST_LDFLAGS"]="-L/usr/lib/x86_64-linux-gnu" S["BOOST_CPPFLAGS"]="-I/usr/include" S["HAVE_CXX11"]="" S["CPP"]="gcc -E" S["LT_SYS_LIBRARY_PATH"]="" S["OTOOL64"]="" S["OTOOL"]="" S["LIPO"]="" S["NMEDIT"]="" S["DSYMUTIL"]="" S["MANIFEST_TOOL"]=":" S["RANLIB"]="ranlib" S["ac_ct_AR"]="ar" S["AR"]="ar" S["DLLTOOL"]="false" S["OBJDUMP"]="objdump" S["LN_S"]="ln -s" S["NM"]="/usr/bin/nm -B" S["ac_ct_DUMPBIN"]="" S["DUMPBIN"]="" S["LD"]="/usr/bin/ld -m elf_x86_64" S["FGREP"]="/bin/grep -F" S["EGREP"]="/bin/grep -E" S["GREP"]="/bin/grep" S["SED"]="/bin/sed" S["am__fastdepCC_FALSE"]="#" S["am__fastdepCC_TRUE"]="" S["CCDEPMODE"]="depmode=gcc3" S["ac_ct_CC"]="gcc" S["CFLAGS"]="-g -O2" S["CC"]="gcc" S["LIBTOOL"]="$(SHELL) $(top_builddir)/libtool" S["host_os"]="linux-gnu" S["host_vendor"]="pc" S["host_cpu"]="x86_64" S["host"]="x86_64-pc-linux-gnu" S["build_os"]="linux-gnu" S["build_vendor"]="pc" S["build_cpu"]="x86_64" S["build"]="x86_64-pc-linux-gnu" S["CXXCPP"]="g++ -E" S["am__fastdepCXX_FALSE"]="#" S["am__fastdepCXX_TRUE"]="" S["CXXDEPMODE"]="depmode=gcc3" S["am__nodep"]="_no" S["AMDEPBACKSLASH"]="\\" S["AMDEP_FALSE"]="#" S["AMDEP_TRUE"]="" S["am__quote"]="" S["am__include"]="include" S["DEPDIR"]=".deps" S["OBJEXT"]="o" S["EXEEXT"]="" S["ac_ct_CXX"]="" S["CPPFLAGS"]="" S["LDFLAGS"]="" S["CXXFLAGS"]=" -std=gnu++11" S["CXX"]="g++" S["AM_BACKSLASH"]="\\" S["AM_DEFAULT_VERBOSITY"]="1" S["AM_DEFAULT_V"]="$(AM_DEFAULT_VERBOSITY)" S["AM_V"]="$(V)" S["am__untar"]="$${TAR-tar} xf -" S["am__tar"]="$${TAR-tar} chof - \"$$tardir\"" S["AMTAR"]="$${TAR-tar}" S["am__leading_dot"]="." S["SET_MAKE"]="" S["AWK"]="gawk" S["mkdir_p"]="$(MKDIR_P)" S["MKDIR_P"]="/bin/mkdir -p" S["INSTALL_STRIP_PROGRAM"]="$(install_sh) -c -s" S["STRIP"]="strip" S["install_sh"]="${SHELL} /home/jcl/programs/vowpal_wabbit/install-sh" S["MAKEINFO"]="${SHELL} /home/jcl/programs/vowpal_wabbit/missing makeinfo" S["AUTOHEADER"]="${SHELL} /home/jcl/programs/vowpal_wabbit/missing autoheader" S["AUTOMAKE"]="${SHELL} /home/jcl/programs/vowpal_wabbit/missing automake-1.15" S["AUTOCONF"]="${SHELL} /home/jcl/programs/vowpal_wabbit/missing autoconf" S["ACLOCAL"]="${SHELL} /home/jcl/programs/vowpal_wabbit/missing aclocal-1.15" S["VERSION"]="8.5.0" S["PACKAGE"]="vowpal_wabbit" S["CYGPATH_W"]="echo" S["am__isrc"]="" S["INSTALL_DATA"]="${INSTALL} -m 644" S["INSTALL_SCRIPT"]="${INSTALL}" S["INSTALL_PROGRAM"]="${INSTALL}" S["target_alias"]="" S["host_alias"]="" S["build_alias"]="" S["LIBS"]="" S["ECHO_T"]="" S["ECHO_N"]="-n" S["ECHO_C"]="" S["DEFS"]="-DHAVE_CONFIG_H" S["mandir"]="${datarootdir}/man" S["localedir"]="${datarootdir}/locale" S["libdir"]="${exec_prefix}/lib" S["psdir"]="${docdir}" S["pdfdir"]="${docdir}" S["dvidir"]="${docdir}" S["htmldir"]="${docdir}" S["infodir"]="${datarootdir}/info" S["docdir"]="${datarootdir}/doc/${PACKAGE_TARNAME}" S["oldincludedir"]="/usr/include" S["includedir"]="${prefix}/include" S["runstatedir"]="${localstatedir}/run" S["localstatedir"]="${prefix}/var" S["sharedstatedir"]="${prefix}/com" S["sysconfdir"]="${prefix}/etc" S["datadir"]="${datarootdir}" S["datarootdir"]="${prefix}/share" S["libexecdir"]="${exec_prefix}/libexec" S["sbindir"]="${exec_prefix}/sbin" S["bindir"]="${exec_prefix}/bin" S["program_transform_name"]="s,x,x," S["prefix"]="/usr/local" S["exec_prefix"]="${prefix}" S["PACKAGE_URL"]="https://github.com/JohnLangford/vowpal_wabbit" S["PACKAGE_BUGREPORT"]="jl@hunch.net" S["PACKAGE_STRING"]="vowpal_wabbit 8.5.0" S["PACKAGE_VERSION"]="8.5.0" S["PACKAGE_TARNAME"]="vowpal_wabbit" S["PACKAGE_NAME"]="vowpal_wabbit" S["PATH_SEPARATOR"]=":" S["SHELL"]="/bin/bash" _ACAWK cat >>"$ac_tmp/subs1.awk" <<_ACAWK && for (key in S) S_is_set[key] = 1 FS = "" } { line = $ 0 nfields = split(line, field, "@") substed = 0 len = length(field[1]) for (i = 2; i < nfields; i++) { key = field[i] keylen = length(key) if (S_is_set[key]) { value = S[key] line = substr(line, 1, len) "" value "" substr(line, len + keylen + 3) len += length(value) + length(field[++i]) substed = 1 } else len += 1 + keylen } print line } _ACAWK if sed "s/$ac_cr//" < /dev/null > /dev/null 2>&1; then sed "s/$ac_cr\$//; s/$ac_cr/$ac_cs_awk_cr/g" else cat fi < "$ac_tmp/subs1.awk" > "$ac_tmp/subs.awk" \ || as_fn_error $? "could not setup config files machinery" "$LINENO" 5 fi # test -n "$CONFIG_FILES" # Set up the scripts for CONFIG_HEADERS section. # No need to generate them if there are no CONFIG_HEADERS. # This happens for instance with `./config.status Makefile'. if test -n "$CONFIG_HEADERS"; then cat >"$ac_tmp/defines.awk" <<\_ACAWK || BEGIN { D["PACKAGE_NAME"]=" \"vowpal_wabbit\"" D["PACKAGE_TARNAME"]=" \"vowpal_wabbit\"" D["PACKAGE_VERSION"]=" \"8.5.0\"" D["PACKAGE_STRING"]=" \"vowpal_wabbit 8.5.0\"" D["PACKAGE_BUGREPORT"]=" \"jl@hunch.net\"" D["PACKAGE_URL"]=" \"https://github.com/JohnLangford/vowpal_wabbit\"" D["PACKAGE"]=" \"vowpal_wabbit\"" D["VERSION"]=" \"8.5.0\"" D["STDC_HEADERS"]=" 1" D["HAVE_SYS_TYPES_H"]=" 1" D["HAVE_SYS_STAT_H"]=" 1" D["HAVE_STDLIB_H"]=" 1" D["HAVE_STRING_H"]=" 1" D["HAVE_MEMORY_H"]=" 1" D["HAVE_STRINGS_H"]=" 1" D["HAVE_INTTYPES_H"]=" 1" D["HAVE_STDINT_H"]=" 1" D["HAVE_UNISTD_H"]=" 1" D["HAVE_DLFCN_H"]=" 1" D["LT_OBJDIR"]=" \".libs/\"" D["STDC_HEADERS"]=" 1" D["HAVE_BOOST"]=" /**/" D["HAVE_BOOST_PROGRAM_OPTIONS"]=" /**/" D["HAVE_PTHREAD_PRIO_INHERIT"]=" 1" D["HAVE_PTHREAD"]=" 1" for (key in D) D_is_set[key] = 1 FS = "" } /^[\t ]*#[\t ]*(define|undef)[\t ]+[_abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ][_abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789]*([\t (]|$)/ { line = $ 0 split(line, arg, " ") if (arg[1] == "#") { defundef = arg[2] mac1 = arg[3] } else { defundef = substr(arg[1], 2) mac1 = arg[2] } split(mac1, mac2, "(") #) macro = mac2[1] prefix = substr(line, 1, index(line, defundef) - 1) if (D_is_set[macro]) { # Preserve the white space surrounding the "#". print prefix "define", macro P[macro] D[macro] next } else { # Replace #undef with comments. This is necessary, for example, # in the case of _POSIX_SOURCE, which is predefined and required # on some systems where configure will not decide to define it. if (defundef == "undef") { print "/*", prefix defundef, macro, "*/" next } } } { print } _ACAWK as_fn_error $? "could not setup config headers machinery" "$LINENO" 5 fi # test -n "$CONFIG_HEADERS" eval set X " :F $CONFIG_FILES :H $CONFIG_HEADERS :C $CONFIG_COMMANDS" shift for ac_tag do case $ac_tag in :[FHLC]) ac_mode=$ac_tag; continue;; esac case $ac_mode$ac_tag in :[FHL]*:*);; :L* | :C*:*) as_fn_error $? "invalid tag \`$ac_tag'" "$LINENO" 5;; :[FH]-) ac_tag=-:-;; :[FH]*) ac_tag=$ac_tag:$ac_tag.in;; esac ac_save_IFS=$IFS IFS=: set x $ac_tag IFS=$ac_save_IFS shift ac_file=$1 shift case $ac_mode in :L) ac_source=$1;; :[FH]) ac_file_inputs= for ac_f do case $ac_f in -) ac_f="$ac_tmp/stdin";; *) # Look for the file first in the build tree, then in the source tree # (if the path is not absolute). The absolute path cannot be DOS-style, # because $ac_f cannot contain `:'. test -f "$ac_f" || case $ac_f in [\\/$]*) false;; *) test -f "$srcdir/$ac_f" && ac_f="$srcdir/$ac_f";; esac || as_fn_error 1 "cannot find input file: \`$ac_f'" "$LINENO" 5;; esac case $ac_f in *\'*) ac_f=`$as_echo "$ac_f" | sed "s/'/'\\\\\\\\''/g"`;; esac as_fn_append ac_file_inputs " '$ac_f'" done # Let's still pretend it is `configure' which instantiates (i.e., don't # use $as_me), people would be surprised to read: # /* config.h. Generated by config.status. */ configure_input='Generated from '` $as_echo "$*" | sed 's|^[^:]*/||;s|:[^:]*/|, |g' `' by configure.' if test x"$ac_file" != x-; then configure_input="$ac_file. $configure_input" { $as_echo "$as_me:${as_lineno-$LINENO}: creating $ac_file" >&5 $as_echo "$as_me: creating $ac_file" >&6;} fi # Neutralize special characters interpreted by sed in replacement strings. case $configure_input in #( *\&* | *\|* | *\\* ) ac_sed_conf_input=`$as_echo "$configure_input" | sed 's/[\\\\&|]/\\\\&/g'`;; #( *) ac_sed_conf_input=$configure_input;; esac case $ac_tag in *:-:* | *:-) cat >"$ac_tmp/stdin" \ || as_fn_error $? "could not create $ac_file" "$LINENO" 5 ;; esac ;; esac ac_dir=`$as_dirname -- "$ac_file" || $as_expr X"$ac_file" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ X"$ac_file" : 'X\(//\)[^/]' \| \ X"$ac_file" : 'X\(//\)$' \| \ X"$ac_file" : 'X\(/\)' \| . 2>/dev/null || $as_echo X"$ac_file" | sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ s//\1/ q } /^X\(\/\/\)[^/].*/{ s//\1/ q } /^X\(\/\/\)$/{ s//\1/ q } /^X\(\/\).*/{ s//\1/ q } s/.*/./; q'` as_dir="$ac_dir"; as_fn_mkdir_p ac_builddir=. case "$ac_dir" in .) ac_dir_suffix= ac_top_builddir_sub=. ac_top_build_prefix= ;; *) ac_dir_suffix=/`$as_echo "$ac_dir" | sed 's|^\.[\\/]||'` # A ".." for each directory in $ac_dir_suffix. ac_top_builddir_sub=`$as_echo "$ac_dir_suffix" | sed 's|/[^\\/]*|/..|g;s|/||'` case $ac_top_builddir_sub in "") ac_top_builddir_sub=. ac_top_build_prefix= ;; *) ac_top_build_prefix=$ac_top_builddir_sub/ ;; esac ;; esac ac_abs_top_builddir=$ac_pwd ac_abs_builddir=$ac_pwd$ac_dir_suffix # for backward compatibility: ac_top_builddir=$ac_top_build_prefix case $srcdir in .) # We are building in place. ac_srcdir=. ac_top_srcdir=$ac_top_builddir_sub ac_abs_top_srcdir=$ac_pwd ;; [\\/]* | ?:[\\/]* ) # Absolute name. ac_srcdir=$srcdir$ac_dir_suffix; ac_top_srcdir=$srcdir ac_abs_top_srcdir=$srcdir ;; *) # Relative name. ac_srcdir=$ac_top_build_prefix$srcdir$ac_dir_suffix ac_top_srcdir=$ac_top_build_prefix$srcdir ac_abs_top_srcdir=$ac_pwd/$srcdir ;; esac ac_abs_srcdir=$ac_abs_top_srcdir$ac_dir_suffix case $ac_mode in :F) # # CONFIG_FILE # case $INSTALL in [\\/$]* | ?:[\\/]* ) ac_INSTALL=$INSTALL ;; *) ac_INSTALL=$ac_top_build_prefix$INSTALL ;; esac ac_MKDIR_P=$MKDIR_P case $MKDIR_P in [\\/$]* | ?:[\\/]* ) ;; */*) ac_MKDIR_P=$ac_top_build_prefix$MKDIR_P ;; esac # If the template does not know about datarootdir, expand it. # FIXME: This hack should be removed a few years after 2.60. ac_datarootdir_hack=; ac_datarootdir_seen= ac_sed_dataroot=' /datarootdir/ { p q } /@datadir@/p /@docdir@/p /@infodir@/p /@localedir@/p /@mandir@/p' case `eval "sed -n \"\$ac_sed_dataroot\" $ac_file_inputs"` in *datarootdir*) ac_datarootdir_seen=yes;; *@datadir@*|*@docdir@*|*@infodir@*|*@localedir@*|*@mandir@*) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $ac_file_inputs seems to ignore the --datarootdir setting" >&5 $as_echo "$as_me: WARNING: $ac_file_inputs seems to ignore the --datarootdir setting" >&2;} ac_datarootdir_hack=' s&@datadir@&${datarootdir}&g s&@docdir@&${datarootdir}/doc/${PACKAGE_TARNAME}&g s&@infodir@&${datarootdir}/info&g s&@localedir@&${datarootdir}/locale&g s&@mandir@&${datarootdir}/man&g s&\${datarootdir}&${prefix}/share&g' ;; esac ac_sed_extra="/^[ ]*VPATH[ ]*=[ ]*/{ h s/// s/^/:/ s/[ ]*$/:/ s/:\$(srcdir):/:/g s/:\${srcdir}:/:/g s/:@srcdir@:/:/g s/^:*// s/:*$// x s/\(=[ ]*\).*/\1/ G s/\n// s/^[^=]*=[ ]*$// } :t /@[a-zA-Z_][a-zA-Z_0-9]*@/!b s|@configure_input@|$ac_sed_conf_input|;t t s&@top_builddir@&$ac_top_builddir_sub&;t t s&@top_build_prefix@&$ac_top_build_prefix&;t t s&@srcdir@&$ac_srcdir&;t t s&@abs_srcdir@&$ac_abs_srcdir&;t t s&@top_srcdir@&$ac_top_srcdir&;t t s&@abs_top_srcdir@&$ac_abs_top_srcdir&;t t s&@builddir@&$ac_builddir&;t t s&@abs_builddir@&$ac_abs_builddir&;t t s&@abs_top_builddir@&$ac_abs_top_builddir&;t t s&@INSTALL@&$ac_INSTALL&;t t s&@MKDIR_P@&$ac_MKDIR_P&;t t $ac_datarootdir_hack " eval sed \"\$ac_sed_extra\" "$ac_file_inputs" | $AWK -f "$ac_tmp/subs.awk" \ >$ac_tmp/out || as_fn_error $? "could not create $ac_file" "$LINENO" 5 test -z "$ac_datarootdir_hack$ac_datarootdir_seen" && { ac_out=`sed -n '/\${datarootdir}/p' "$ac_tmp/out"`; test -n "$ac_out"; } && { ac_out=`sed -n '/^[ ]*datarootdir[ ]*:*=/p' \ "$ac_tmp/out"`; test -z "$ac_out"; } && { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $ac_file contains a reference to the variable \`datarootdir' which seems to be undefined. Please make sure it is defined" >&5 $as_echo "$as_me: WARNING: $ac_file contains a reference to the variable \`datarootdir' which seems to be undefined. Please make sure it is defined" >&2;} rm -f "$ac_tmp/stdin" case $ac_file in -) cat "$ac_tmp/out" && rm -f "$ac_tmp/out";; *) rm -f "$ac_file" && mv "$ac_tmp/out" "$ac_file";; esac \ || as_fn_error $? "could not create $ac_file" "$LINENO" 5 ;; :H) # # CONFIG_HEADER # if test x"$ac_file" != x-; then { $as_echo "/* $configure_input */" \ && eval '$AWK -f "$ac_tmp/defines.awk"' "$ac_file_inputs" } >"$ac_tmp/config.h" \ || as_fn_error $? "could not create $ac_file" "$LINENO" 5 if diff "$ac_file" "$ac_tmp/config.h" >/dev/null 2>&1; then { $as_echo "$as_me:${as_lineno-$LINENO}: $ac_file is unchanged" >&5 $as_echo "$as_me: $ac_file is unchanged" >&6;} else rm -f "$ac_file" mv "$ac_tmp/config.h" "$ac_file" \ || as_fn_error $? "could not create $ac_file" "$LINENO" 5 fi else $as_echo "/* $configure_input */" \ && eval '$AWK -f "$ac_tmp/defines.awk"' "$ac_file_inputs" \ || as_fn_error $? "could not create -" "$LINENO" 5 fi # Compute "$ac_file"'s index in $config_headers. _am_arg="$ac_file" _am_stamp_count=1 for _am_header in $config_headers :; do case $_am_header in $_am_arg | $_am_arg:* ) break ;; * ) _am_stamp_count=`expr $_am_stamp_count + 1` ;; esac done echo "timestamp for $_am_arg" >`$as_dirname -- "$_am_arg" || $as_expr X"$_am_arg" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ X"$_am_arg" : 'X\(//\)[^/]' \| \ X"$_am_arg" : 'X\(//\)$' \| \ X"$_am_arg" : 'X\(/\)' \| . 2>/dev/null || $as_echo X"$_am_arg" | sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ s//\1/ q } /^X\(\/\/\)[^/].*/{ s//\1/ q } /^X\(\/\/\)$/{ s//\1/ q } /^X\(\/\).*/{ s//\1/ q } s/.*/./; q'`/stamp-h$_am_stamp_count ;; :C) { $as_echo "$as_me:${as_lineno-$LINENO}: executing $ac_file commands" >&5 $as_echo "$as_me: executing $ac_file commands" >&6;} ;; esac case $ac_file$ac_mode in "depfiles":C) test x"$AMDEP_TRUE" != x"" || { # Older Autoconf quotes --file arguments for eval, but not when files # are listed without --file. Let's play safe and only enable the eval # if we detect the quoting. case $CONFIG_FILES in *\'*) eval set x "$CONFIG_FILES" ;; *) set x $CONFIG_FILES ;; esac shift for mf do # Strip MF so we end up with the name of the file. mf=`echo "$mf" | sed -e 's/:.*$//'` # Check whether this is an Automake generated Makefile or not. # We used to match only the files named 'Makefile.in', but # some people rename them; so instead we look at the file content. # Grep'ing the first line is not enough: some people post-process # each Makefile.in and add a new line on top of each file to say so. # Grep'ing the whole file is not good either: AIX grep has a line # limit of 2048, but all sed's we know have understand at least 4000. if sed -n 's,^#.*generated by automake.*,X,p' "$mf" | grep X >/dev/null 2>&1; then dirpart=`$as_dirname -- "$mf" || $as_expr X"$mf" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ X"$mf" : 'X\(//\)[^/]' \| \ X"$mf" : 'X\(//\)$' \| \ X"$mf" : 'X\(/\)' \| . 2>/dev/null || $as_echo X"$mf" | sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ s//\1/ q } /^X\(\/\/\)[^/].*/{ s//\1/ q } /^X\(\/\/\)$/{ s//\1/ q } /^X\(\/\).*/{ s//\1/ q } s/.*/./; q'` else continue fi # Extract the definition of DEPDIR, am__include, and am__quote # from the Makefile without running 'make'. DEPDIR=`sed -n 's/^DEPDIR = //p' < "$mf"` test -z "$DEPDIR" && continue am__include=`sed -n 's/^am__include = //p' < "$mf"` test -z "$am__include" && continue am__quote=`sed -n 's/^am__quote = //p' < "$mf"` # Find all dependency output files, they are included files with # $(DEPDIR) in their names. We invoke sed twice because it is the # simplest approach to changing $(DEPDIR) to its actual value in the # expansion. for file in `sed -n " s/^$am__include $am__quote\(.*(DEPDIR).*\)$am__quote"'$/\1/p' <"$mf" | \ sed -e 's/\$(DEPDIR)/'"$DEPDIR"'/g'`; do # Make sure the directory exists. test -f "$dirpart/$file" && continue fdir=`$as_dirname -- "$file" || $as_expr X"$file" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ X"$file" : 'X\(//\)[^/]' \| \ X"$file" : 'X\(//\)$' \| \ X"$file" : 'X\(/\)' \| . 2>/dev/null || $as_echo X"$file" | sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ s//\1/ q } /^X\(\/\/\)[^/].*/{ s//\1/ q } /^X\(\/\/\)$/{ s//\1/ q } /^X\(\/\).*/{ s//\1/ q } s/.*/./; q'` as_dir=$dirpart/$fdir; as_fn_mkdir_p # echo "creating $dirpart/$file" echo '# dummy' > "$dirpart/$file" done done } ;; "libtool":C) # See if we are running on zsh, and set the options that allow our # commands through without removal of \ escapes. if test -n "${ZSH_VERSION+set}"; then setopt NO_GLOB_SUBST fi cfgfile=${ofile}T trap "$RM \"$cfgfile\"; exit 1" 1 2 15 $RM "$cfgfile" cat <<_LT_EOF >> "$cfgfile" #! $SHELL # Generated automatically by $as_me ($PACKAGE) $VERSION # Libtool was configured on host `(hostname || uname -n) 2>/dev/null | sed 1q`: # NOTE: Changes made to this file will be lost: look at ltmain.sh. # Provide generalized library-building support services. # Written by Gordon Matzigkeit, 1996 # Copyright (C) 2014 Free Software Foundation, Inc. # This is free software; see the source for copying conditions. There is NO # warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # GNU Libtool is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of of the License, or # (at your option) any later version. # # As a special exception to the GNU General Public License, if you # distribute this file as part of a program or library that is built # using GNU Libtool, you may include this file under the same # distribution terms that you use for the rest of that program. # # GNU Libtool is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # The names of the tagged configurations supported by this script. available_tags='CXX ' # Configured defaults for sys_lib_dlsearch_path munging. : \${LT_SYS_LIBRARY_PATH="$configure_time_lt_sys_library_path"} # ### BEGIN LIBTOOL CONFIG # Which release of libtool.m4 was used? macro_version=$macro_version macro_revision=$macro_revision # Whether or not to build shared libraries. build_libtool_libs=$enable_shared # Whether or not to build static libraries. build_old_libs=$enable_static # What type of objects to build. pic_mode=$pic_mode # Whether or not to optimize for fast installation. fast_install=$enable_fast_install # Shared archive member basename,for filename based shared library versioning on AIX. shared_archive_member_spec=$shared_archive_member_spec # Shell to use when invoking shell scripts. SHELL=$lt_SHELL # An echo program that protects backslashes. ECHO=$lt_ECHO # The PATH separator for the build system. PATH_SEPARATOR=$lt_PATH_SEPARATOR # The host system. host_alias=$host_alias host=$host host_os=$host_os # The build system. build_alias=$build_alias build=$build build_os=$build_os # A sed program that does not truncate output. SED=$lt_SED # Sed that helps us avoid accidentally triggering echo(1) options like -n. Xsed="\$SED -e 1s/^X//" # A grep program that handles long lines. GREP=$lt_GREP # An ERE matcher. EGREP=$lt_EGREP # A literal string matcher. FGREP=$lt_FGREP # A BSD- or MS-compatible name lister. NM=$lt_NM # Whether we need soft or hard links. LN_S=$lt_LN_S # What is the maximum length of a command? max_cmd_len=$max_cmd_len # Object file suffix (normally "o"). objext=$ac_objext # Executable file suffix (normally ""). exeext=$exeext # whether the shell understands "unset". lt_unset=$lt_unset # turn spaces into newlines. SP2NL=$lt_lt_SP2NL # turn newlines into spaces. NL2SP=$lt_lt_NL2SP # convert \$build file names to \$host format. to_host_file_cmd=$lt_cv_to_host_file_cmd # convert \$build files to toolchain format. to_tool_file_cmd=$lt_cv_to_tool_file_cmd # An object symbol dumper. OBJDUMP=$lt_OBJDUMP # Method to check whether dependent libraries are shared objects. deplibs_check_method=$lt_deplibs_check_method # Command to use when deplibs_check_method = "file_magic". file_magic_cmd=$lt_file_magic_cmd # How to find potential files when deplibs_check_method = "file_magic". file_magic_glob=$lt_file_magic_glob # Find potential files using nocaseglob when deplibs_check_method = "file_magic". want_nocaseglob=$lt_want_nocaseglob # DLL creation program. DLLTOOL=$lt_DLLTOOL # Command to associate shared and link libraries. sharedlib_from_linklib_cmd=$lt_sharedlib_from_linklib_cmd # The archiver. AR=$lt_AR # Flags to create an archive. AR_FLAGS=$lt_AR_FLAGS # How to feed a file listing to the archiver. archiver_list_spec=$lt_archiver_list_spec # A symbol stripping program. STRIP=$lt_STRIP # Commands used to install an old-style archive. RANLIB=$lt_RANLIB old_postinstall_cmds=$lt_old_postinstall_cmds old_postuninstall_cmds=$lt_old_postuninstall_cmds # Whether to use a lock for old archive extraction. lock_old_archive_extraction=$lock_old_archive_extraction # A C compiler. LTCC=$lt_CC # LTCC compiler flags. LTCFLAGS=$lt_CFLAGS # Take the output of nm and produce a listing of raw symbols and C names. global_symbol_pipe=$lt_lt_cv_sys_global_symbol_pipe # Transform the output of nm in a proper C declaration. global_symbol_to_cdecl=$lt_lt_cv_sys_global_symbol_to_cdecl # Transform the output of nm into a list of symbols to manually relocate. global_symbol_to_import=$lt_lt_cv_sys_global_symbol_to_import # Transform the output of nm in a C name address pair. global_symbol_to_c_name_address=$lt_lt_cv_sys_global_symbol_to_c_name_address # Transform the output of nm in a C name address pair when lib prefix is needed. global_symbol_to_c_name_address_lib_prefix=$lt_lt_cv_sys_global_symbol_to_c_name_address_lib_prefix # The name lister interface. nm_interface=$lt_lt_cv_nm_interface # Specify filename containing input files for \$NM. nm_file_list_spec=$lt_nm_file_list_spec # The root where to search for dependent libraries,and where our libraries should be installed. lt_sysroot=$lt_sysroot # Command to truncate a binary pipe. lt_truncate_bin=$lt_lt_cv_truncate_bin # The name of the directory that contains temporary libtool files. objdir=$objdir # Used to examine libraries when file_magic_cmd begins with "file". MAGIC_CMD=$MAGIC_CMD # Must we lock files when doing compilation? need_locks=$lt_need_locks # Manifest tool. MANIFEST_TOOL=$lt_MANIFEST_TOOL # Tool to manipulate archived DWARF debug symbol files on Mac OS X. DSYMUTIL=$lt_DSYMUTIL # Tool to change global to local symbols on Mac OS X. NMEDIT=$lt_NMEDIT # Tool to manipulate fat objects and archives on Mac OS X. LIPO=$lt_LIPO # ldd/readelf like tool for Mach-O binaries on Mac OS X. OTOOL=$lt_OTOOL # ldd/readelf like tool for 64 bit Mach-O binaries on Mac OS X 10.4. OTOOL64=$lt_OTOOL64 # Old archive suffix (normally "a"). libext=$libext # Shared library suffix (normally ".so"). shrext_cmds=$lt_shrext_cmds # The commands to extract the exported symbol list from a shared archive. extract_expsyms_cmds=$lt_extract_expsyms_cmds # Variables whose values should be saved in libtool wrapper scripts and # restored at link time. variables_saved_for_relink=$lt_variables_saved_for_relink # Do we need the "lib" prefix for modules? need_lib_prefix=$need_lib_prefix # Do we need a version for libraries? need_version=$need_version # Library versioning type. version_type=$version_type # Shared library runtime path variable. runpath_var=$runpath_var # Shared library path variable. shlibpath_var=$shlibpath_var # Is shlibpath searched before the hard-coded library search path? shlibpath_overrides_runpath=$shlibpath_overrides_runpath # Format of library name prefix. libname_spec=$lt_libname_spec # List of archive names. First name is the real one, the rest are links. # The last name is the one that the linker finds with -lNAME library_names_spec=$lt_library_names_spec # The coded name of the library, if different from the real name. soname_spec=$lt_soname_spec # Permission mode override for installation of shared libraries. install_override_mode=$lt_install_override_mode # Command to use after installation of a shared archive. postinstall_cmds=$lt_postinstall_cmds # Command to use after uninstallation of a shared archive. postuninstall_cmds=$lt_postuninstall_cmds # Commands used to finish a libtool library installation in a directory. finish_cmds=$lt_finish_cmds # As "finish_cmds", except a single script fragment to be evaled but # not shown. finish_eval=$lt_finish_eval # Whether we should hardcode library paths into libraries. hardcode_into_libs=$hardcode_into_libs # Compile-time system search path for libraries. sys_lib_search_path_spec=$lt_sys_lib_search_path_spec # Detected run-time system search path for libraries. sys_lib_dlsearch_path_spec=$lt_configure_time_dlsearch_path # Explicit LT_SYS_LIBRARY_PATH set during ./configure time. configure_time_lt_sys_library_path=$lt_configure_time_lt_sys_library_path # Whether dlopen is supported. dlopen_support=$enable_dlopen # Whether dlopen of programs is supported. dlopen_self=$enable_dlopen_self # Whether dlopen of statically linked programs is supported. dlopen_self_static=$enable_dlopen_self_static # Commands to strip libraries. old_striplib=$lt_old_striplib striplib=$lt_striplib # The linker used to build libraries. LD=$lt_LD # How to create reloadable object files. reload_flag=$lt_reload_flag reload_cmds=$lt_reload_cmds # Commands used to build an old-style archive. old_archive_cmds=$lt_old_archive_cmds # A language specific compiler. CC=$lt_compiler # Is the compiler the GNU compiler? with_gcc=$GCC # Compiler flag to turn off builtin functions. no_builtin_flag=$lt_lt_prog_compiler_no_builtin_flag # Additional compiler flags for building library objects. pic_flag=$lt_lt_prog_compiler_pic # How to pass a linker flag through the compiler. wl=$lt_lt_prog_compiler_wl # Compiler flag to prevent dynamic linking. link_static_flag=$lt_lt_prog_compiler_static # Does compiler simultaneously support -c and -o options? compiler_c_o=$lt_lt_cv_prog_compiler_c_o # Whether or not to add -lc for building shared libraries. build_libtool_need_lc=$archive_cmds_need_lc # Whether or not to disallow shared libs when runtime libs are static. allow_libtool_libs_with_static_runtimes=$enable_shared_with_static_runtimes # Compiler flag to allow reflexive dlopens. export_dynamic_flag_spec=$lt_export_dynamic_flag_spec # Compiler flag to generate shared objects directly from archives. whole_archive_flag_spec=$lt_whole_archive_flag_spec # Whether the compiler copes with passing no objects directly. compiler_needs_object=$lt_compiler_needs_object # Create an old-style archive from a shared archive. old_archive_from_new_cmds=$lt_old_archive_from_new_cmds # Create a temporary old-style archive to link instead of a shared archive. old_archive_from_expsyms_cmds=$lt_old_archive_from_expsyms_cmds # Commands used to build a shared archive. archive_cmds=$lt_archive_cmds archive_expsym_cmds=$lt_archive_expsym_cmds # Commands used to build a loadable module if different from building # a shared archive. module_cmds=$lt_module_cmds module_expsym_cmds=$lt_module_expsym_cmds # Whether we are building with GNU ld or not. with_gnu_ld=$lt_with_gnu_ld # Flag that allows shared libraries with undefined symbols to be built. allow_undefined_flag=$lt_allow_undefined_flag # Flag that enforces no undefined symbols. no_undefined_flag=$lt_no_undefined_flag # Flag to hardcode \$libdir into a binary during linking. # This must work even if \$libdir does not exist hardcode_libdir_flag_spec=$lt_hardcode_libdir_flag_spec # Whether we need a single "-rpath" flag with a separated argument. hardcode_libdir_separator=$lt_hardcode_libdir_separator # Set to "yes" if using DIR/libNAME\$shared_ext during linking hardcodes # DIR into the resulting binary. hardcode_direct=$hardcode_direct # Set to "yes" if using DIR/libNAME\$shared_ext during linking hardcodes # DIR into the resulting binary and the resulting library dependency is # "absolute",i.e impossible to change by setting \$shlibpath_var if the # library is relocated. hardcode_direct_absolute=$hardcode_direct_absolute # Set to "yes" if using the -LDIR flag during linking hardcodes DIR # into the resulting binary. hardcode_minus_L=$hardcode_minus_L # Set to "yes" if using SHLIBPATH_VAR=DIR during linking hardcodes DIR # into the resulting binary. hardcode_shlibpath_var=$hardcode_shlibpath_var # Set to "yes" if building a shared library automatically hardcodes DIR # into the library and all subsequent libraries and executables linked # against it. hardcode_automatic=$hardcode_automatic # Set to yes if linker adds runtime paths of dependent libraries # to runtime path list. inherit_rpath=$inherit_rpath # Whether libtool must link a program against all its dependency libraries. link_all_deplibs=$link_all_deplibs # Set to "yes" if exported symbols are required. always_export_symbols=$always_export_symbols # The commands to list exported symbols. export_symbols_cmds=$lt_export_symbols_cmds # Symbols that should not be listed in the preloaded symbols. exclude_expsyms=$lt_exclude_expsyms # Symbols that must always be exported. include_expsyms=$lt_include_expsyms # Commands necessary for linking programs (against libraries) with templates. prelink_cmds=$lt_prelink_cmds # Commands necessary for finishing linking programs. postlink_cmds=$lt_postlink_cmds # Specify filename containing input files. file_list_spec=$lt_file_list_spec # How to hardcode a shared library path into an executable. hardcode_action=$hardcode_action # The directories searched by this compiler when creating a shared library. compiler_lib_search_dirs=$lt_compiler_lib_search_dirs # Dependencies to place before and after the objects being linked to # create a shared library. predep_objects=$lt_predep_objects postdep_objects=$lt_postdep_objects predeps=$lt_predeps postdeps=$lt_postdeps # The library search path used internally by the compiler when linking # a shared library. compiler_lib_search_path=$lt_compiler_lib_search_path # ### END LIBTOOL CONFIG _LT_EOF cat <<'_LT_EOF' >> "$cfgfile" # ### BEGIN FUNCTIONS SHARED WITH CONFIGURE # func_munge_path_list VARIABLE PATH # ----------------------------------- # VARIABLE is name of variable containing _space_ separated list of # directories to be munged by the contents of PATH, which is string # having a format: # "DIR[:DIR]:" # string "DIR[ DIR]" will be prepended to VARIABLE # ":DIR[:DIR]" # string "DIR[ DIR]" will be appended to VARIABLE # "DIRP[:DIRP]::[DIRA:]DIRA" # string "DIRP[ DIRP]" will be prepended to VARIABLE and string # "DIRA[ DIRA]" will be appended to VARIABLE # "DIR[:DIR]" # VARIABLE will be replaced by "DIR[ DIR]" func_munge_path_list () { case x$2 in x) ;; *:) eval $1=\"`$ECHO $2 | $SED 's/:/ /g'` \$$1\" ;; x:*) eval $1=\"\$$1 `$ECHO $2 | $SED 's/:/ /g'`\" ;; *::*) eval $1=\"\$$1\ `$ECHO $2 | $SED -e 's/.*:://' -e 's/:/ /g'`\" eval $1=\"`$ECHO $2 | $SED -e 's/::.*//' -e 's/:/ /g'`\ \$$1\" ;; *) eval $1=\"`$ECHO $2 | $SED 's/:/ /g'`\" ;; esac } # Calculate cc_basename. Skip known compiler wrappers and cross-prefix. func_cc_basename () { for cc_temp in $*""; do case $cc_temp in compile | *[\\/]compile | ccache | *[\\/]ccache ) ;; distcc | *[\\/]distcc | purify | *[\\/]purify ) ;; \-*) ;; *) break;; esac done func_cc_basename_result=`$ECHO "$cc_temp" | $SED "s%.*/%%; s%^$host_alias-%%"` } # ### END FUNCTIONS SHARED WITH CONFIGURE _LT_EOF case $host_os in aix3*) cat <<\_LT_EOF >> "$cfgfile" # AIX sometimes has problems with the GCC collect2 program. For some # reason, if we set the COLLECT_NAMES environment variable, the problems # vanish in a puff of smoke. if test set != "${COLLECT_NAMES+set}"; then COLLECT_NAMES= export COLLECT_NAMES fi _LT_EOF ;; esac ltmain=$ac_aux_dir/ltmain.sh # We use sed instead of cat because bash on DJGPP gets confused if # if finds mixed CR/LF and LF-only lines. Since sed operates in # text mode, it properly converts lines to CR/LF. This bash problem # is reportedly fixed, but why not run on old versions too? sed '$q' "$ltmain" >> "$cfgfile" \ || (rm -f "$cfgfile"; exit 1) mv -f "$cfgfile" "$ofile" || (rm -f "$ofile" && cp "$cfgfile" "$ofile" && rm -f "$cfgfile") chmod +x "$ofile" cat <<_LT_EOF >> "$ofile" # ### BEGIN LIBTOOL TAG CONFIG: CXX # The linker used to build libraries. LD=$lt_LD_CXX # How to create reloadable object files. reload_flag=$lt_reload_flag_CXX reload_cmds=$lt_reload_cmds_CXX # Commands used to build an old-style archive. old_archive_cmds=$lt_old_archive_cmds_CXX # A language specific compiler. CC=$lt_compiler_CXX # Is the compiler the GNU compiler? with_gcc=$GCC_CXX # Compiler flag to turn off builtin functions. no_builtin_flag=$lt_lt_prog_compiler_no_builtin_flag_CXX # Additional compiler flags for building library objects. pic_flag=$lt_lt_prog_compiler_pic_CXX # How to pass a linker flag through the compiler. wl=$lt_lt_prog_compiler_wl_CXX # Compiler flag to prevent dynamic linking. link_static_flag=$lt_lt_prog_compiler_static_CXX # Does compiler simultaneously support -c and -o options? compiler_c_o=$lt_lt_cv_prog_compiler_c_o_CXX # Whether or not to add -lc for building shared libraries. build_libtool_need_lc=$archive_cmds_need_lc_CXX # Whether or not to disallow shared libs when runtime libs are static. allow_libtool_libs_with_static_runtimes=$enable_shared_with_static_runtimes_CXX # Compiler flag to allow reflexive dlopens. export_dynamic_flag_spec=$lt_export_dynamic_flag_spec_CXX # Compiler flag to generate shared objects directly from archives. whole_archive_flag_spec=$lt_whole_archive_flag_spec_CXX # Whether the compiler copes with passing no objects directly. compiler_needs_object=$lt_compiler_needs_object_CXX # Create an old-style archive from a shared archive. old_archive_from_new_cmds=$lt_old_archive_from_new_cmds_CXX # Create a temporary old-style archive to link instead of a shared archive. old_archive_from_expsyms_cmds=$lt_old_archive_from_expsyms_cmds_CXX # Commands used to build a shared archive. archive_cmds=$lt_archive_cmds_CXX archive_expsym_cmds=$lt_archive_expsym_cmds_CXX # Commands used to build a loadable module if different from building # a shared archive. module_cmds=$lt_module_cmds_CXX module_expsym_cmds=$lt_module_expsym_cmds_CXX # Whether we are building with GNU ld or not. with_gnu_ld=$lt_with_gnu_ld_CXX # Flag that allows shared libraries with undefined symbols to be built. allow_undefined_flag=$lt_allow_undefined_flag_CXX # Flag that enforces no undefined symbols. no_undefined_flag=$lt_no_undefined_flag_CXX # Flag to hardcode \$libdir into a binary during linking. # This must work even if \$libdir does not exist hardcode_libdir_flag_spec=$lt_hardcode_libdir_flag_spec_CXX # Whether we need a single "-rpath" flag with a separated argument. hardcode_libdir_separator=$lt_hardcode_libdir_separator_CXX # Set to "yes" if using DIR/libNAME\$shared_ext during linking hardcodes # DIR into the resulting binary. hardcode_direct=$hardcode_direct_CXX # Set to "yes" if using DIR/libNAME\$shared_ext during linking hardcodes # DIR into the resulting binary and the resulting library dependency is # "absolute",i.e impossible to change by setting \$shlibpath_var if the # library is relocated. hardcode_direct_absolute=$hardcode_direct_absolute_CXX # Set to "yes" if using the -LDIR flag during linking hardcodes DIR # into the resulting binary. hardcode_minus_L=$hardcode_minus_L_CXX # Set to "yes" if using SHLIBPATH_VAR=DIR during linking hardcodes DIR # into the resulting binary. hardcode_shlibpath_var=$hardcode_shlibpath_var_CXX # Set to "yes" if building a shared library automatically hardcodes DIR # into the library and all subsequent libraries and executables linked # against it. hardcode_automatic=$hardcode_automatic_CXX # Set to yes if linker adds runtime paths of dependent libraries # to runtime path list. inherit_rpath=$inherit_rpath_CXX # Whether libtool must link a program against all its dependency libraries. link_all_deplibs=$link_all_deplibs_CXX # Set to "yes" if exported symbols are required. always_export_symbols=$always_export_symbols_CXX # The commands to list exported symbols. export_symbols_cmds=$lt_export_symbols_cmds_CXX # Symbols that should not be listed in the preloaded symbols. exclude_expsyms=$lt_exclude_expsyms_CXX # Symbols that must always be exported. include_expsyms=$lt_include_expsyms_CXX # Commands necessary for linking programs (against libraries) with templates. prelink_cmds=$lt_prelink_cmds_CXX # Commands necessary for finishing linking programs. postlink_cmds=$lt_postlink_cmds_CXX # Specify filename containing input files. file_list_spec=$lt_file_list_spec_CXX # How to hardcode a shared library path into an executable. hardcode_action=$hardcode_action_CXX # The directories searched by this compiler when creating a shared library. compiler_lib_search_dirs=$lt_compiler_lib_search_dirs_CXX # Dependencies to place before and after the objects being linked to # create a shared library. predep_objects=$lt_predep_objects_CXX postdep_objects=$lt_postdep_objects_CXX predeps=$lt_predeps_CXX postdeps=$lt_postdeps_CXX # The library search path used internally by the compiler when linking # a shared library. compiler_lib_search_path=$lt_compiler_lib_search_path_CXX # ### END LIBTOOL TAG CONFIG: CXX _LT_EOF ;; esac done # for ac_tag as_fn_exit 0 vowpal-wabbit-8.5.0.dfsg1/config.sub000077500000000000000000001064601323713302100172720ustar00rootroot00000000000000#! /bin/sh # Configuration validation subroutine script. # Copyright 1992-2015 Free Software Foundation, Inc. timestamp='2015-08-20' # This file is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU # General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, see . # # As a special exception to the GNU General Public License, if you # distribute this file as part of a program that contains a # configuration script generated by Autoconf, you may include it under # the same distribution terms that you use for the rest of that # program. This Exception is an additional permission under section 7 # of the GNU General Public License, version 3 ("GPLv3"). # Please send patches to . # # Configuration subroutine to validate and canonicalize a configuration type. # Supply the specified configuration type as an argument. # If it is invalid, we print an error message on stderr and exit with code 1. # Otherwise, we print the canonical config type on stdout and succeed. # You can get the latest version of this script from: # http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub;hb=HEAD # This file is supposed to be the same for all GNU packages # and recognize all the CPU types, system types and aliases # that are meaningful with *any* GNU software. # Each package is responsible for reporting which valid configurations # it does not support. The user should be able to distinguish # a failure to support a valid configuration from a meaningless # configuration. # The goal of this file is to map all the various variations of a given # machine specification into a single specification in the form: # CPU_TYPE-MANUFACTURER-OPERATING_SYSTEM # or in some cases, the newer four-part form: # CPU_TYPE-MANUFACTURER-KERNEL-OPERATING_SYSTEM # It is wrong to echo any other type of specification. me=`echo "$0" | sed -e 's,.*/,,'` usage="\ Usage: $0 [OPTION] CPU-MFR-OPSYS $0 [OPTION] ALIAS Canonicalize a configuration name. Operation modes: -h, --help print this help, then exit -t, --time-stamp print date of last modification, then exit -v, --version print version number, then exit Report bugs and patches to ." version="\ GNU config.sub ($timestamp) Copyright 1992-2015 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." help=" Try \`$me --help' for more information." # Parse command line while test $# -gt 0 ; do case $1 in --time-stamp | --time* | -t ) echo "$timestamp" ; exit ;; --version | -v ) echo "$version" ; exit ;; --help | --h* | -h ) echo "$usage"; exit ;; -- ) # Stop option processing shift; break ;; - ) # Use stdin as input. break ;; -* ) echo "$me: invalid option $1$help" exit 1 ;; *local*) # First pass through any local machine types. echo $1 exit ;; * ) break ;; esac done case $# in 0) echo "$me: missing argument$help" >&2 exit 1;; 1) ;; *) echo "$me: too many arguments$help" >&2 exit 1;; esac # Separate what the user gave into CPU-COMPANY and OS or KERNEL-OS (if any). # Here we must recognize all the valid KERNEL-OS combinations. maybe_os=`echo $1 | sed 's/^\(.*\)-\([^-]*-[^-]*\)$/\2/'` case $maybe_os in nto-qnx* | linux-gnu* | linux-android* | linux-dietlibc | linux-newlib* | \ linux-musl* | linux-uclibc* | uclinux-uclibc* | uclinux-gnu* | kfreebsd*-gnu* | \ knetbsd*-gnu* | netbsd*-gnu* | netbsd*-eabi* | \ kopensolaris*-gnu* | \ storm-chaos* | os2-emx* | rtmk-nova*) os=-$maybe_os basic_machine=`echo $1 | sed 's/^\(.*\)-\([^-]*-[^-]*\)$/\1/'` ;; android-linux) os=-linux-android basic_machine=`echo $1 | sed 's/^\(.*\)-\([^-]*-[^-]*\)$/\1/'`-unknown ;; *) basic_machine=`echo $1 | sed 's/-[^-]*$//'` if [ $basic_machine != $1 ] then os=`echo $1 | sed 's/.*-/-/'` else os=; fi ;; esac ### Let's recognize common machines as not being operating systems so ### that things like config.sub decstation-3100 work. We also ### recognize some manufacturers as not being operating systems, so we ### can provide default operating systems below. case $os in -sun*os*) # Prevent following clause from handling this invalid input. ;; -dec* | -mips* | -sequent* | -encore* | -pc532* | -sgi* | -sony* | \ -att* | -7300* | -3300* | -delta* | -motorola* | -sun[234]* | \ -unicom* | -ibm* | -next | -hp | -isi* | -apollo | -altos* | \ -convergent* | -ncr* | -news | -32* | -3600* | -3100* | -hitachi* |\ -c[123]* | -convex* | -sun | -crds | -omron* | -dg | -ultra | -tti* | \ -harris | -dolphin | -highlevel | -gould | -cbm | -ns | -masscomp | \ -apple | -axis | -knuth | -cray | -microblaze*) os= basic_machine=$1 ;; -bluegene*) os=-cnk ;; -sim | -cisco | -oki | -wec | -winbond) os= basic_machine=$1 ;; -scout) ;; -wrs) os=-vxworks basic_machine=$1 ;; -chorusos*) os=-chorusos basic_machine=$1 ;; -chorusrdb) os=-chorusrdb basic_machine=$1 ;; -hiux*) os=-hiuxwe2 ;; -sco6) os=-sco5v6 basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` ;; -sco5) os=-sco3.2v5 basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` ;; -sco4) os=-sco3.2v4 basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` ;; -sco3.2.[4-9]*) os=`echo $os | sed -e 's/sco3.2./sco3.2v/'` basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` ;; -sco3.2v[4-9]*) # Don't forget version if it is 3.2v4 or newer. basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` ;; -sco5v6*) # Don't forget version if it is 3.2v4 or newer. basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` ;; -sco*) os=-sco3.2v2 basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` ;; -udk*) basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` ;; -isc) os=-isc2.2 basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` ;; -clix*) basic_machine=clipper-intergraph ;; -isc*) basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` ;; -lynx*178) os=-lynxos178 ;; -lynx*5) os=-lynxos5 ;; -lynx*) os=-lynxos ;; -ptx*) basic_machine=`echo $1 | sed -e 's/86-.*/86-sequent/'` ;; -windowsnt*) os=`echo $os | sed -e 's/windowsnt/winnt/'` ;; -psos*) os=-psos ;; -mint | -mint[0-9]*) basic_machine=m68k-atari os=-mint ;; esac # Decode aliases for certain CPU-COMPANY combinations. case $basic_machine in # Recognize the basic CPU types without company name. # Some are omitted here because they have special meanings below. 1750a | 580 \ | a29k \ | aarch64 | aarch64_be \ | alpha | alphaev[4-8] | alphaev56 | alphaev6[78] | alphapca5[67] \ | alpha64 | alpha64ev[4-8] | alpha64ev56 | alpha64ev6[78] | alpha64pca5[67] \ | am33_2.0 \ | arc | arceb \ | arm | arm[bl]e | arme[lb] | armv[2-8] | armv[3-8][lb] | armv7[arm] \ | avr | avr32 \ | ba \ | be32 | be64 \ | bfin \ | c4x | c8051 | clipper \ | d10v | d30v | dlx | dsp16xx \ | e2k | epiphany \ | fido | fr30 | frv | ft32 \ | h8300 | h8500 | hppa | hppa1.[01] | hppa2.0 | hppa2.0[nw] | hppa64 \ | hexagon \ | i370 | i860 | i960 | ia64 \ | ip2k | iq2000 \ | k1om \ | le32 | le64 \ | lm32 \ | m32c | m32r | m32rle | m68000 | m68k | m88k \ | maxq | mb | microblaze | microblazeel | mcore | mep | metag \ | mips | mipsbe | mipseb | mipsel | mipsle \ | mips16 \ | mips64 | mips64el \ | mips64octeon | mips64octeonel \ | mips64orion | mips64orionel \ | mips64r5900 | mips64r5900el \ | mips64vr | mips64vrel \ | mips64vr4100 | mips64vr4100el \ | mips64vr4300 | mips64vr4300el \ | mips64vr5000 | mips64vr5000el \ | mips64vr5900 | mips64vr5900el \ | mipsisa32 | mipsisa32el \ | mipsisa32r2 | mipsisa32r2el \ | mipsisa32r6 | mipsisa32r6el \ | mipsisa64 | mipsisa64el \ | mipsisa64r2 | mipsisa64r2el \ | mipsisa64r6 | mipsisa64r6el \ | mipsisa64sb1 | mipsisa64sb1el \ | mipsisa64sr71k | mipsisa64sr71kel \ | mipsr5900 | mipsr5900el \ | mipstx39 | mipstx39el \ | mn10200 | mn10300 \ | moxie \ | mt \ | msp430 \ | nds32 | nds32le | nds32be \ | nios | nios2 | nios2eb | nios2el \ | ns16k | ns32k \ | open8 | or1k | or1knd | or32 \ | pdp10 | pdp11 | pj | pjl \ | powerpc | powerpc64 | powerpc64le | powerpcle \ | pyramid \ | riscv32 | riscv64 \ | rl78 | rx \ | score \ | sh | sh[1234] | sh[24]a | sh[24]aeb | sh[23]e | sh[234]eb | sheb | shbe | shle | sh[1234]le | sh3ele \ | sh64 | sh64le \ | sparc | sparc64 | sparc64b | sparc64v | sparc86x | sparclet | sparclite \ | sparcv8 | sparcv9 | sparcv9b | sparcv9v \ | spu \ | tahoe | tic4x | tic54x | tic55x | tic6x | tic80 | tron \ | ubicom32 \ | v850 | v850e | v850e1 | v850e2 | v850es | v850e2v3 \ | visium \ | we32k \ | x86 | xc16x | xstormy16 | xtensa \ | z8k | z80) basic_machine=$basic_machine-unknown ;; c54x) basic_machine=tic54x-unknown ;; c55x) basic_machine=tic55x-unknown ;; c6x) basic_machine=tic6x-unknown ;; leon|leon[3-9]) basic_machine=sparc-$basic_machine ;; m6811 | m68hc11 | m6812 | m68hc12 | m68hcs12x | nvptx | picochip) basic_machine=$basic_machine-unknown os=-none ;; m88110 | m680[12346]0 | m683?2 | m68360 | m5200 | v70 | w65 | z8k) ;; ms1) basic_machine=mt-unknown ;; strongarm | thumb | xscale) basic_machine=arm-unknown ;; xgate) basic_machine=$basic_machine-unknown os=-none ;; xscaleeb) basic_machine=armeb-unknown ;; xscaleel) basic_machine=armel-unknown ;; # We use `pc' rather than `unknown' # because (1) that's what they normally are, and # (2) the word "unknown" tends to confuse beginning users. i*86 | x86_64) basic_machine=$basic_machine-pc ;; # Object if more than one company name word. *-*-*) echo Invalid configuration \`$1\': machine \`$basic_machine\' not recognized 1>&2 exit 1 ;; # Recognize the basic CPU types with company name. 580-* \ | a29k-* \ | aarch64-* | aarch64_be-* \ | alpha-* | alphaev[4-8]-* | alphaev56-* | alphaev6[78]-* \ | alpha64-* | alpha64ev[4-8]-* | alpha64ev56-* | alpha64ev6[78]-* \ | alphapca5[67]-* | alpha64pca5[67]-* | arc-* | arceb-* \ | arm-* | armbe-* | armle-* | armeb-* | armv*-* \ | avr-* | avr32-* \ | ba-* \ | be32-* | be64-* \ | bfin-* | bs2000-* \ | c[123]* | c30-* | [cjt]90-* | c4x-* \ | c8051-* | clipper-* | craynv-* | cydra-* \ | d10v-* | d30v-* | dlx-* \ | e2k-* | elxsi-* \ | f30[01]-* | f700-* | fido-* | fr30-* | frv-* | fx80-* \ | h8300-* | h8500-* \ | hppa-* | hppa1.[01]-* | hppa2.0-* | hppa2.0[nw]-* | hppa64-* \ | hexagon-* \ | i*86-* | i860-* | i960-* | ia64-* \ | ip2k-* | iq2000-* \ | k1om-* \ | le32-* | le64-* \ | lm32-* \ | m32c-* | m32r-* | m32rle-* \ | m68000-* | m680[012346]0-* | m68360-* | m683?2-* | m68k-* \ | m88110-* | m88k-* | maxq-* | mcore-* | metag-* \ | microblaze-* | microblazeel-* \ | mips-* | mipsbe-* | mipseb-* | mipsel-* | mipsle-* \ | mips16-* \ | mips64-* | mips64el-* \ | mips64octeon-* | mips64octeonel-* \ | mips64orion-* | mips64orionel-* \ | mips64r5900-* | mips64r5900el-* \ | mips64vr-* | mips64vrel-* \ | mips64vr4100-* | mips64vr4100el-* \ | mips64vr4300-* | mips64vr4300el-* \ | mips64vr5000-* | mips64vr5000el-* \ | mips64vr5900-* | mips64vr5900el-* \ | mipsisa32-* | mipsisa32el-* \ | mipsisa32r2-* | mipsisa32r2el-* \ | mipsisa32r6-* | mipsisa32r6el-* \ | mipsisa64-* | mipsisa64el-* \ | mipsisa64r2-* | mipsisa64r2el-* \ | mipsisa64r6-* | mipsisa64r6el-* \ | mipsisa64sb1-* | mipsisa64sb1el-* \ | mipsisa64sr71k-* | mipsisa64sr71kel-* \ | mipsr5900-* | mipsr5900el-* \ | mipstx39-* | mipstx39el-* \ | mmix-* \ | mt-* \ | msp430-* \ | nds32-* | nds32le-* | nds32be-* \ | nios-* | nios2-* | nios2eb-* | nios2el-* \ | none-* | np1-* | ns16k-* | ns32k-* \ | open8-* \ | or1k*-* \ | orion-* \ | pdp10-* | pdp11-* | pj-* | pjl-* | pn-* | power-* \ | powerpc-* | powerpc64-* | powerpc64le-* | powerpcle-* \ | pyramid-* \ | riscv32-* | riscv64-* \ | rl78-* | romp-* | rs6000-* | rx-* \ | sh-* | sh[1234]-* | sh[24]a-* | sh[24]aeb-* | sh[23]e-* | sh[34]eb-* | sheb-* | shbe-* \ | shle-* | sh[1234]le-* | sh3ele-* | sh64-* | sh64le-* \ | sparc-* | sparc64-* | sparc64b-* | sparc64v-* | sparc86x-* | sparclet-* \ | sparclite-* \ | sparcv8-* | sparcv9-* | sparcv9b-* | sparcv9v-* | sv1-* | sx*-* \ | tahoe-* \ | tic30-* | tic4x-* | tic54x-* | tic55x-* | tic6x-* | tic80-* \ | tile*-* \ | tron-* \ | ubicom32-* \ | v850-* | v850e-* | v850e1-* | v850es-* | v850e2-* | v850e2v3-* \ | vax-* \ | visium-* \ | we32k-* \ | x86-* | x86_64-* | xc16x-* | xps100-* \ | xstormy16-* | xtensa*-* \ | ymp-* \ | z8k-* | z80-*) ;; # Recognize the basic CPU types without company name, with glob match. xtensa*) basic_machine=$basic_machine-unknown ;; # Recognize the various machine names and aliases which stand # for a CPU type and a company and sometimes even an OS. 386bsd) basic_machine=i386-unknown os=-bsd ;; 3b1 | 7300 | 7300-att | att-7300 | pc7300 | safari | unixpc) basic_machine=m68000-att ;; 3b*) basic_machine=we32k-att ;; a29khif) basic_machine=a29k-amd os=-udi ;; abacus) basic_machine=abacus-unknown ;; adobe68k) basic_machine=m68010-adobe os=-scout ;; alliant | fx80) basic_machine=fx80-alliant ;; altos | altos3068) basic_machine=m68k-altos ;; am29k) basic_machine=a29k-none os=-bsd ;; amd64) basic_machine=x86_64-pc ;; amd64-*) basic_machine=x86_64-`echo $basic_machine | sed 's/^[^-]*-//'` ;; amdahl) basic_machine=580-amdahl os=-sysv ;; amiga | amiga-*) basic_machine=m68k-unknown ;; amigaos | amigados) basic_machine=m68k-unknown os=-amigaos ;; amigaunix | amix) basic_machine=m68k-unknown os=-sysv4 ;; apollo68) basic_machine=m68k-apollo os=-sysv ;; apollo68bsd) basic_machine=m68k-apollo os=-bsd ;; aros) basic_machine=i386-pc os=-aros ;; asmjs) basic_machine=asmjs-unknown ;; aux) basic_machine=m68k-apple os=-aux ;; balance) basic_machine=ns32k-sequent os=-dynix ;; blackfin) basic_machine=bfin-unknown os=-linux ;; blackfin-*) basic_machine=bfin-`echo $basic_machine | sed 's/^[^-]*-//'` os=-linux ;; bluegene*) basic_machine=powerpc-ibm os=-cnk ;; c54x-*) basic_machine=tic54x-`echo $basic_machine | sed 's/^[^-]*-//'` ;; c55x-*) basic_machine=tic55x-`echo $basic_machine | sed 's/^[^-]*-//'` ;; c6x-*) basic_machine=tic6x-`echo $basic_machine | sed 's/^[^-]*-//'` ;; c90) basic_machine=c90-cray os=-unicos ;; cegcc) basic_machine=arm-unknown os=-cegcc ;; convex-c1) basic_machine=c1-convex os=-bsd ;; convex-c2) basic_machine=c2-convex os=-bsd ;; convex-c32) basic_machine=c32-convex os=-bsd ;; convex-c34) basic_machine=c34-convex os=-bsd ;; convex-c38) basic_machine=c38-convex os=-bsd ;; cray | j90) basic_machine=j90-cray os=-unicos ;; craynv) basic_machine=craynv-cray os=-unicosmp ;; cr16 | cr16-*) basic_machine=cr16-unknown os=-elf ;; crds | unos) basic_machine=m68k-crds ;; crisv32 | crisv32-* | etraxfs*) basic_machine=crisv32-axis ;; cris | cris-* | etrax*) basic_machine=cris-axis ;; crx) basic_machine=crx-unknown os=-elf ;; da30 | da30-*) basic_machine=m68k-da30 ;; decstation | decstation-3100 | pmax | pmax-* | pmin | dec3100 | decstatn) basic_machine=mips-dec ;; decsystem10* | dec10*) basic_machine=pdp10-dec os=-tops10 ;; decsystem20* | dec20*) basic_machine=pdp10-dec os=-tops20 ;; delta | 3300 | motorola-3300 | motorola-delta \ | 3300-motorola | delta-motorola) basic_machine=m68k-motorola ;; delta88) basic_machine=m88k-motorola os=-sysv3 ;; dicos) basic_machine=i686-pc os=-dicos ;; djgpp) basic_machine=i586-pc os=-msdosdjgpp ;; dpx20 | dpx20-*) basic_machine=rs6000-bull os=-bosx ;; dpx2* | dpx2*-bull) basic_machine=m68k-bull os=-sysv3 ;; ebmon29k) basic_machine=a29k-amd os=-ebmon ;; elxsi) basic_machine=elxsi-elxsi os=-bsd ;; encore | umax | mmax) basic_machine=ns32k-encore ;; es1800 | OSE68k | ose68k | ose | OSE) basic_machine=m68k-ericsson os=-ose ;; fx2800) basic_machine=i860-alliant ;; genix) basic_machine=ns32k-ns ;; gmicro) basic_machine=tron-gmicro os=-sysv ;; go32) basic_machine=i386-pc os=-go32 ;; h3050r* | hiux*) basic_machine=hppa1.1-hitachi os=-hiuxwe2 ;; h8300hms) basic_machine=h8300-hitachi os=-hms ;; h8300xray) basic_machine=h8300-hitachi os=-xray ;; h8500hms) basic_machine=h8500-hitachi os=-hms ;; harris) basic_machine=m88k-harris os=-sysv3 ;; hp300-*) basic_machine=m68k-hp ;; hp300bsd) basic_machine=m68k-hp os=-bsd ;; hp300hpux) basic_machine=m68k-hp os=-hpux ;; hp3k9[0-9][0-9] | hp9[0-9][0-9]) basic_machine=hppa1.0-hp ;; hp9k2[0-9][0-9] | hp9k31[0-9]) basic_machine=m68000-hp ;; hp9k3[2-9][0-9]) basic_machine=m68k-hp ;; hp9k6[0-9][0-9] | hp6[0-9][0-9]) basic_machine=hppa1.0-hp ;; hp9k7[0-79][0-9] | hp7[0-79][0-9]) basic_machine=hppa1.1-hp ;; hp9k78[0-9] | hp78[0-9]) # FIXME: really hppa2.0-hp basic_machine=hppa1.1-hp ;; hp9k8[67]1 | hp8[67]1 | hp9k80[24] | hp80[24] | hp9k8[78]9 | hp8[78]9 | hp9k893 | hp893) # FIXME: really hppa2.0-hp basic_machine=hppa1.1-hp ;; hp9k8[0-9][13679] | hp8[0-9][13679]) basic_machine=hppa1.1-hp ;; hp9k8[0-9][0-9] | hp8[0-9][0-9]) basic_machine=hppa1.0-hp ;; hppa-next) os=-nextstep3 ;; hppaosf) basic_machine=hppa1.1-hp os=-osf ;; hppro) basic_machine=hppa1.1-hp os=-proelf ;; i370-ibm* | ibm*) basic_machine=i370-ibm ;; i*86v32) basic_machine=`echo $1 | sed -e 's/86.*/86-pc/'` os=-sysv32 ;; i*86v4*) basic_machine=`echo $1 | sed -e 's/86.*/86-pc/'` os=-sysv4 ;; i*86v) basic_machine=`echo $1 | sed -e 's/86.*/86-pc/'` os=-sysv ;; i*86sol2) basic_machine=`echo $1 | sed -e 's/86.*/86-pc/'` os=-solaris2 ;; i386mach) basic_machine=i386-mach os=-mach ;; i386-vsta | vsta) basic_machine=i386-unknown os=-vsta ;; iris | iris4d) basic_machine=mips-sgi case $os in -irix*) ;; *) os=-irix4 ;; esac ;; isi68 | isi) basic_machine=m68k-isi os=-sysv ;; leon-*|leon[3-9]-*) basic_machine=sparc-`echo $basic_machine | sed 's/-.*//'` ;; m68knommu) basic_machine=m68k-unknown os=-linux ;; m68knommu-*) basic_machine=m68k-`echo $basic_machine | sed 's/^[^-]*-//'` os=-linux ;; m88k-omron*) basic_machine=m88k-omron ;; magnum | m3230) basic_machine=mips-mips os=-sysv ;; merlin) basic_machine=ns32k-utek os=-sysv ;; microblaze*) basic_machine=microblaze-xilinx ;; mingw64) basic_machine=x86_64-pc os=-mingw64 ;; mingw32) basic_machine=i686-pc os=-mingw32 ;; mingw32ce) basic_machine=arm-unknown os=-mingw32ce ;; miniframe) basic_machine=m68000-convergent ;; *mint | -mint[0-9]* | *MiNT | *MiNT[0-9]*) basic_machine=m68k-atari os=-mint ;; mips3*-*) basic_machine=`echo $basic_machine | sed -e 's/mips3/mips64/'` ;; mips3*) basic_machine=`echo $basic_machine | sed -e 's/mips3/mips64/'`-unknown ;; monitor) basic_machine=m68k-rom68k os=-coff ;; morphos) basic_machine=powerpc-unknown os=-morphos ;; moxiebox) basic_machine=moxie-unknown os=-moxiebox ;; msdos) basic_machine=i386-pc os=-msdos ;; ms1-*) basic_machine=`echo $basic_machine | sed -e 's/ms1-/mt-/'` ;; msys) basic_machine=i686-pc os=-msys ;; mvs) basic_machine=i370-ibm os=-mvs ;; nacl) basic_machine=le32-unknown os=-nacl ;; ncr3000) basic_machine=i486-ncr os=-sysv4 ;; netbsd386) basic_machine=i386-unknown os=-netbsd ;; netwinder) basic_machine=armv4l-rebel os=-linux ;; news | news700 | news800 | news900) basic_machine=m68k-sony os=-newsos ;; news1000) basic_machine=m68030-sony os=-newsos ;; news-3600 | risc-news) basic_machine=mips-sony os=-newsos ;; necv70) basic_machine=v70-nec os=-sysv ;; next | m*-next ) basic_machine=m68k-next case $os in -nextstep* ) ;; -ns2*) os=-nextstep2 ;; *) os=-nextstep3 ;; esac ;; nh3000) basic_machine=m68k-harris os=-cxux ;; nh[45]000) basic_machine=m88k-harris os=-cxux ;; nindy960) basic_machine=i960-intel os=-nindy ;; mon960) basic_machine=i960-intel os=-mon960 ;; nonstopux) basic_machine=mips-compaq os=-nonstopux ;; np1) basic_machine=np1-gould ;; neo-tandem) basic_machine=neo-tandem ;; nse-tandem) basic_machine=nse-tandem ;; nsr-tandem) basic_machine=nsr-tandem ;; op50n-* | op60c-*) basic_machine=hppa1.1-oki os=-proelf ;; openrisc | openrisc-*) basic_machine=or32-unknown ;; os400) basic_machine=powerpc-ibm os=-os400 ;; OSE68000 | ose68000) basic_machine=m68000-ericsson os=-ose ;; os68k) basic_machine=m68k-none os=-os68k ;; pa-hitachi) basic_machine=hppa1.1-hitachi os=-hiuxwe2 ;; paragon) basic_machine=i860-intel os=-osf ;; parisc) basic_machine=hppa-unknown os=-linux ;; parisc-*) basic_machine=hppa-`echo $basic_machine | sed 's/^[^-]*-//'` os=-linux ;; pbd) basic_machine=sparc-tti ;; pbb) basic_machine=m68k-tti ;; pc532 | pc532-*) basic_machine=ns32k-pc532 ;; pc98) basic_machine=i386-pc ;; pc98-*) basic_machine=i386-`echo $basic_machine | sed 's/^[^-]*-//'` ;; pentium | p5 | k5 | k6 | nexgen | viac3) basic_machine=i586-pc ;; pentiumpro | p6 | 6x86 | athlon | athlon_*) basic_machine=i686-pc ;; pentiumii | pentium2 | pentiumiii | pentium3) basic_machine=i686-pc ;; pentium4) basic_machine=i786-pc ;; pentium-* | p5-* | k5-* | k6-* | nexgen-* | viac3-*) basic_machine=i586-`echo $basic_machine | sed 's/^[^-]*-//'` ;; pentiumpro-* | p6-* | 6x86-* | athlon-*) basic_machine=i686-`echo $basic_machine | sed 's/^[^-]*-//'` ;; pentiumii-* | pentium2-* | pentiumiii-* | pentium3-*) basic_machine=i686-`echo $basic_machine | sed 's/^[^-]*-//'` ;; pentium4-*) basic_machine=i786-`echo $basic_machine | sed 's/^[^-]*-//'` ;; pn) basic_machine=pn-gould ;; power) basic_machine=power-ibm ;; ppc | ppcbe) basic_machine=powerpc-unknown ;; ppc-* | ppcbe-*) basic_machine=powerpc-`echo $basic_machine | sed 's/^[^-]*-//'` ;; ppcle | powerpclittle | ppc-le | powerpc-little) basic_machine=powerpcle-unknown ;; ppcle-* | powerpclittle-*) basic_machine=powerpcle-`echo $basic_machine | sed 's/^[^-]*-//'` ;; ppc64) basic_machine=powerpc64-unknown ;; ppc64-*) basic_machine=powerpc64-`echo $basic_machine | sed 's/^[^-]*-//'` ;; ppc64le | powerpc64little | ppc64-le | powerpc64-little) basic_machine=powerpc64le-unknown ;; ppc64le-* | powerpc64little-*) basic_machine=powerpc64le-`echo $basic_machine | sed 's/^[^-]*-//'` ;; ps2) basic_machine=i386-ibm ;; pw32) basic_machine=i586-unknown os=-pw32 ;; rdos | rdos64) basic_machine=x86_64-pc os=-rdos ;; rdos32) basic_machine=i386-pc os=-rdos ;; rom68k) basic_machine=m68k-rom68k os=-coff ;; rm[46]00) basic_machine=mips-siemens ;; rtpc | rtpc-*) basic_machine=romp-ibm ;; s390 | s390-*) basic_machine=s390-ibm ;; s390x | s390x-*) basic_machine=s390x-ibm ;; sa29200) basic_machine=a29k-amd os=-udi ;; sb1) basic_machine=mipsisa64sb1-unknown ;; sb1el) basic_machine=mipsisa64sb1el-unknown ;; sde) basic_machine=mipsisa32-sde os=-elf ;; sei) basic_machine=mips-sei os=-seiux ;; sequent) basic_machine=i386-sequent ;; sh) basic_machine=sh-hitachi os=-hms ;; sh5el) basic_machine=sh5le-unknown ;; sh64) basic_machine=sh64-unknown ;; sparclite-wrs | simso-wrs) basic_machine=sparclite-wrs os=-vxworks ;; sps7) basic_machine=m68k-bull os=-sysv2 ;; spur) basic_machine=spur-unknown ;; st2000) basic_machine=m68k-tandem ;; stratus) basic_machine=i860-stratus os=-sysv4 ;; strongarm-* | thumb-*) basic_machine=arm-`echo $basic_machine | sed 's/^[^-]*-//'` ;; sun2) basic_machine=m68000-sun ;; sun2os3) basic_machine=m68000-sun os=-sunos3 ;; sun2os4) basic_machine=m68000-sun os=-sunos4 ;; sun3os3) basic_machine=m68k-sun os=-sunos3 ;; sun3os4) basic_machine=m68k-sun os=-sunos4 ;; sun4os3) basic_machine=sparc-sun os=-sunos3 ;; sun4os4) basic_machine=sparc-sun os=-sunos4 ;; sun4sol2) basic_machine=sparc-sun os=-solaris2 ;; sun3 | sun3-*) basic_machine=m68k-sun ;; sun4) basic_machine=sparc-sun ;; sun386 | sun386i | roadrunner) basic_machine=i386-sun ;; sv1) basic_machine=sv1-cray os=-unicos ;; symmetry) basic_machine=i386-sequent os=-dynix ;; t3e) basic_machine=alphaev5-cray os=-unicos ;; t90) basic_machine=t90-cray os=-unicos ;; tile*) basic_machine=$basic_machine-unknown os=-linux-gnu ;; tx39) basic_machine=mipstx39-unknown ;; tx39el) basic_machine=mipstx39el-unknown ;; toad1) basic_machine=pdp10-xkl os=-tops20 ;; tower | tower-32) basic_machine=m68k-ncr ;; tpf) basic_machine=s390x-ibm os=-tpf ;; udi29k) basic_machine=a29k-amd os=-udi ;; ultra3) basic_machine=a29k-nyu os=-sym1 ;; v810 | necv810) basic_machine=v810-nec os=-none ;; vaxv) basic_machine=vax-dec os=-sysv ;; vms) basic_machine=vax-dec os=-vms ;; vpp*|vx|vx-*) basic_machine=f301-fujitsu ;; vxworks960) basic_machine=i960-wrs os=-vxworks ;; vxworks68) basic_machine=m68k-wrs os=-vxworks ;; vxworks29k) basic_machine=a29k-wrs os=-vxworks ;; w65*) basic_machine=w65-wdc os=-none ;; w89k-*) basic_machine=hppa1.1-winbond os=-proelf ;; xbox) basic_machine=i686-pc os=-mingw32 ;; xps | xps100) basic_machine=xps100-honeywell ;; xscale-* | xscalee[bl]-*) basic_machine=`echo $basic_machine | sed 's/^xscale/arm/'` ;; ymp) basic_machine=ymp-cray os=-unicos ;; z8k-*-coff) basic_machine=z8k-unknown os=-sim ;; z80-*-coff) basic_machine=z80-unknown os=-sim ;; none) basic_machine=none-none os=-none ;; # Here we handle the default manufacturer of certain CPU types. It is in # some cases the only manufacturer, in others, it is the most popular. w89k) basic_machine=hppa1.1-winbond ;; op50n) basic_machine=hppa1.1-oki ;; op60c) basic_machine=hppa1.1-oki ;; romp) basic_machine=romp-ibm ;; mmix) basic_machine=mmix-knuth ;; rs6000) basic_machine=rs6000-ibm ;; vax) basic_machine=vax-dec ;; pdp10) # there are many clones, so DEC is not a safe bet basic_machine=pdp10-unknown ;; pdp11) basic_machine=pdp11-dec ;; we32k) basic_machine=we32k-att ;; sh[1234] | sh[24]a | sh[24]aeb | sh[34]eb | sh[1234]le | sh[23]ele) basic_machine=sh-unknown ;; sparc | sparcv8 | sparcv9 | sparcv9b | sparcv9v) basic_machine=sparc-sun ;; cydra) basic_machine=cydra-cydrome ;; orion) basic_machine=orion-highlevel ;; orion105) basic_machine=clipper-highlevel ;; mac | mpw | mac-mpw) basic_machine=m68k-apple ;; pmac | pmac-mpw) basic_machine=powerpc-apple ;; *-unknown) # Make sure to match an already-canonicalized machine name. ;; *) echo Invalid configuration \`$1\': machine \`$basic_machine\' not recognized 1>&2 exit 1 ;; esac # Here we canonicalize certain aliases for manufacturers. case $basic_machine in *-digital*) basic_machine=`echo $basic_machine | sed 's/digital.*/dec/'` ;; *-commodore*) basic_machine=`echo $basic_machine | sed 's/commodore.*/cbm/'` ;; *) ;; esac # Decode manufacturer-specific aliases for certain operating systems. if [ x"$os" != x"" ] then case $os in # First match some system type aliases # that might get confused with valid system types. # -solaris* is a basic system type, with this one exception. -auroraux) os=-auroraux ;; -solaris1 | -solaris1.*) os=`echo $os | sed -e 's|solaris1|sunos4|'` ;; -solaris) os=-solaris2 ;; -svr4*) os=-sysv4 ;; -unixware*) os=-sysv4.2uw ;; -gnu/linux*) os=`echo $os | sed -e 's|gnu/linux|linux-gnu|'` ;; # First accept the basic system types. # The portable systems comes first. # Each alternative MUST END IN A *, to match a version number. # -sysv* is not here because it comes later, after sysvr4. -gnu* | -bsd* | -mach* | -minix* | -genix* | -ultrix* | -irix* \ | -*vms* | -sco* | -esix* | -isc* | -aix* | -cnk* | -sunos | -sunos[34]*\ | -hpux* | -unos* | -osf* | -luna* | -dgux* | -auroraux* | -solaris* \ | -sym* | -kopensolaris* | -plan9* \ | -amigaos* | -amigados* | -msdos* | -newsos* | -unicos* | -aof* \ | -aos* | -aros* | -cloudabi* | -sortix* \ | -nindy* | -vxsim* | -vxworks* | -ebmon* | -hms* | -mvs* \ | -clix* | -riscos* | -uniplus* | -iris* | -rtu* | -xenix* \ | -hiux* | -386bsd* | -knetbsd* | -mirbsd* | -netbsd* \ | -bitrig* | -openbsd* | -solidbsd* \ | -ekkobsd* | -kfreebsd* | -freebsd* | -riscix* | -lynxos* \ | -bosx* | -nextstep* | -cxux* | -aout* | -elf* | -oabi* \ | -ptx* | -coff* | -ecoff* | -winnt* | -domain* | -vsta* \ | -udi* | -eabi* | -lites* | -ieee* | -go32* | -aux* \ | -chorusos* | -chorusrdb* | -cegcc* \ | -cygwin* | -msys* | -pe* | -psos* | -moss* | -proelf* | -rtems* \ | -mingw32* | -mingw64* | -linux-gnu* | -linux-android* \ | -linux-newlib* | -linux-musl* | -linux-uclibc* \ | -uxpv* | -beos* | -mpeix* | -udk* | -moxiebox* \ | -interix* | -uwin* | -mks* | -rhapsody* | -darwin* | -opened* \ | -openstep* | -oskit* | -conix* | -pw32* | -nonstopux* \ | -storm-chaos* | -tops10* | -tenex* | -tops20* | -its* \ | -os2* | -vos* | -palmos* | -uclinux* | -nucleus* \ | -morphos* | -superux* | -rtmk* | -rtmk-nova* | -windiss* \ | -powermax* | -dnix* | -nx6 | -nx7 | -sei* | -dragonfly* \ | -skyos* | -haiku* | -rdos* | -toppers* | -drops* | -es* | -tirtos*) # Remember, each alternative MUST END IN *, to match a version number. ;; -qnx*) case $basic_machine in x86-* | i*86-*) ;; *) os=-nto$os ;; esac ;; -nto-qnx*) ;; -nto*) os=`echo $os | sed -e 's|nto|nto-qnx|'` ;; -sim | -es1800* | -hms* | -xray | -os68k* | -none* | -v88r* \ | -windows* | -osx | -abug | -netware* | -os9* | -beos* | -haiku* \ | -macos* | -mpw* | -magic* | -mmixware* | -mon960* | -lnews*) ;; -mac*) os=`echo $os | sed -e 's|mac|macos|'` ;; -linux-dietlibc) os=-linux-dietlibc ;; -linux*) os=`echo $os | sed -e 's|linux|linux-gnu|'` ;; -sunos5*) os=`echo $os | sed -e 's|sunos5|solaris2|'` ;; -sunos6*) os=`echo $os | sed -e 's|sunos6|solaris3|'` ;; -opened*) os=-openedition ;; -os400*) os=-os400 ;; -wince*) os=-wince ;; -osfrose*) os=-osfrose ;; -osf*) os=-osf ;; -utek*) os=-bsd ;; -dynix*) os=-bsd ;; -acis*) os=-aos ;; -atheos*) os=-atheos ;; -syllable*) os=-syllable ;; -386bsd) os=-bsd ;; -ctix* | -uts*) os=-sysv ;; -nova*) os=-rtmk-nova ;; -ns2 ) os=-nextstep2 ;; -nsk*) os=-nsk ;; # Preserve the version number of sinix5. -sinix5.*) os=`echo $os | sed -e 's|sinix|sysv|'` ;; -sinix*) os=-sysv4 ;; -tpf*) os=-tpf ;; -triton*) os=-sysv3 ;; -oss*) os=-sysv3 ;; -svr4) os=-sysv4 ;; -svr3) os=-sysv3 ;; -sysvr4) os=-sysv4 ;; # This must come after -sysvr4. -sysv*) ;; -ose*) os=-ose ;; -es1800*) os=-ose ;; -xenix) os=-xenix ;; -*mint | -mint[0-9]* | -*MiNT | -MiNT[0-9]*) os=-mint ;; -aros*) os=-aros ;; -zvmoe) os=-zvmoe ;; -dicos*) os=-dicos ;; -nacl*) ;; -none) ;; *) # Get rid of the `-' at the beginning of $os. os=`echo $os | sed 's/[^-]*-//'` echo Invalid configuration \`$1\': system \`$os\' not recognized 1>&2 exit 1 ;; esac else # Here we handle the default operating systems that come with various machines. # The value should be what the vendor currently ships out the door with their # machine or put another way, the most popular os provided with the machine. # Note that if you're going to try to match "-MANUFACTURER" here (say, # "-sun"), then you have to tell the case statement up towards the top # that MANUFACTURER isn't an operating system. Otherwise, code above # will signal an error saying that MANUFACTURER isn't an operating # system, and we'll never get to this point. case $basic_machine in score-*) os=-elf ;; spu-*) os=-elf ;; *-acorn) os=-riscix1.2 ;; arm*-rebel) os=-linux ;; arm*-semi) os=-aout ;; c4x-* | tic4x-*) os=-coff ;; c8051-*) os=-elf ;; hexagon-*) os=-elf ;; tic54x-*) os=-coff ;; tic55x-*) os=-coff ;; tic6x-*) os=-coff ;; # This must come before the *-dec entry. pdp10-*) os=-tops20 ;; pdp11-*) os=-none ;; *-dec | vax-*) os=-ultrix4.2 ;; m68*-apollo) os=-domain ;; i386-sun) os=-sunos4.0.2 ;; m68000-sun) os=-sunos3 ;; m68*-cisco) os=-aout ;; mep-*) os=-elf ;; mips*-cisco) os=-elf ;; mips*-*) os=-elf ;; or32-*) os=-coff ;; *-tti) # must be before sparc entry or we get the wrong os. os=-sysv3 ;; sparc-* | *-sun) os=-sunos4.1.1 ;; *-be) os=-beos ;; *-haiku) os=-haiku ;; *-ibm) os=-aix ;; *-knuth) os=-mmixware ;; *-wec) os=-proelf ;; *-winbond) os=-proelf ;; *-oki) os=-proelf ;; *-hp) os=-hpux ;; *-hitachi) os=-hiux ;; i860-* | *-att | *-ncr | *-altos | *-motorola | *-convergent) os=-sysv ;; *-cbm) os=-amigaos ;; *-dg) os=-dgux ;; *-dolphin) os=-sysv3 ;; m68k-ccur) os=-rtu ;; m88k-omron*) os=-luna ;; *-next ) os=-nextstep ;; *-sequent) os=-ptx ;; *-crds) os=-unos ;; *-ns) os=-genix ;; i370-*) os=-mvs ;; *-next) os=-nextstep3 ;; *-gould) os=-sysv ;; *-highlevel) os=-bsd ;; *-encore) os=-bsd ;; *-sgi) os=-irix ;; *-siemens) os=-sysv4 ;; *-masscomp) os=-rtu ;; f30[01]-fujitsu | f700-fujitsu) os=-uxpv ;; *-rom68k) os=-coff ;; *-*bug) os=-coff ;; *-apple) os=-macos ;; *-atari*) os=-mint ;; *) os=-none ;; esac fi # Here we handle the case where we know the os, and the CPU type, but not the # manufacturer. We pick the logical manufacturer. vendor=unknown case $basic_machine in *-unknown) case $os in -riscix*) vendor=acorn ;; -sunos*) vendor=sun ;; -cnk*|-aix*) vendor=ibm ;; -beos*) vendor=be ;; -hpux*) vendor=hp ;; -mpeix*) vendor=hp ;; -hiux*) vendor=hitachi ;; -unos*) vendor=crds ;; -dgux*) vendor=dg ;; -luna*) vendor=omron ;; -genix*) vendor=ns ;; -mvs* | -opened*) vendor=ibm ;; -os400*) vendor=ibm ;; -ptx*) vendor=sequent ;; -tpf*) vendor=ibm ;; -vxsim* | -vxworks* | -windiss*) vendor=wrs ;; -aux*) vendor=apple ;; -hms*) vendor=hitachi ;; -mpw* | -macos*) vendor=apple ;; -*mint | -mint[0-9]* | -*MiNT | -MiNT[0-9]*) vendor=atari ;; -vos*) vendor=stratus ;; esac basic_machine=`echo $basic_machine | sed "s/unknown/$vendor/"` ;; esac echo $basic_machine$os exit # Local variables: # eval: (add-hook 'write-file-hooks 'time-stamp) # time-stamp-start: "timestamp='" # time-stamp-format: "%:y-%02m-%02d" # time-stamp-end: "'" # End: vowpal-wabbit-8.5.0.dfsg1/configure000077500000000000000000023204471323713302100172230ustar00rootroot00000000000000#! /bin/sh # Guess values for system-dependent variables and create Makefiles. # Generated by GNU Autoconf 2.69 for vowpal_wabbit 8.5.0. # # Report bugs to . # # # Copyright (C) 1992-1996, 1998-2012 Free Software Foundation, Inc. # # # This configure script is free software; the Free Software Foundation # gives unlimited permission to copy, distribute and modify it. ## -------------------- ## ## M4sh Initialization. ## ## -------------------- ## # Be more Bourne compatible DUALCASE=1; export DUALCASE # for MKS sh if test -n "${ZSH_VERSION+set}" && (emulate sh) >/dev/null 2>&1; then : emulate sh NULLCMD=: # Pre-4.2 versions of Zsh do word splitting on ${1+"$@"}, which # is contrary to our usage. Disable this feature. alias -g '${1+"$@"}'='"$@"' setopt NO_GLOB_SUBST else case `(set -o) 2>/dev/null` in #( *posix*) : set -o posix ;; #( *) : ;; esac fi as_nl=' ' export as_nl # Printing a long string crashes Solaris 7 /usr/bin/printf. as_echo='\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\' as_echo=$as_echo$as_echo$as_echo$as_echo$as_echo as_echo=$as_echo$as_echo$as_echo$as_echo$as_echo$as_echo # Prefer a ksh shell builtin over an external printf program on Solaris, # but without wasting forks for bash or zsh. if test -z "$BASH_VERSION$ZSH_VERSION" \ && (test "X`print -r -- $as_echo`" = "X$as_echo") 2>/dev/null; then as_echo='print -r --' as_echo_n='print -rn --' elif (test "X`printf %s $as_echo`" = "X$as_echo") 2>/dev/null; then as_echo='printf %s\n' as_echo_n='printf %s' else if test "X`(/usr/ucb/echo -n -n $as_echo) 2>/dev/null`" = "X-n $as_echo"; then as_echo_body='eval /usr/ucb/echo -n "$1$as_nl"' as_echo_n='/usr/ucb/echo -n' else as_echo_body='eval expr "X$1" : "X\\(.*\\)"' as_echo_n_body='eval arg=$1; case $arg in #( *"$as_nl"*) expr "X$arg" : "X\\(.*\\)$as_nl"; arg=`expr "X$arg" : ".*$as_nl\\(.*\\)"`;; esac; expr "X$arg" : "X\\(.*\\)" | tr -d "$as_nl" ' export as_echo_n_body as_echo_n='sh -c $as_echo_n_body as_echo' fi export as_echo_body as_echo='sh -c $as_echo_body as_echo' fi # The user is always right. if test "${PATH_SEPARATOR+set}" != set; then PATH_SEPARATOR=: (PATH='/bin;/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 && { (PATH='/bin:/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 || PATH_SEPARATOR=';' } fi # IFS # We need space, tab and new line, in precisely that order. Quoting is # there to prevent editors from complaining about space-tab. # (If _AS_PATH_WALK were called with IFS unset, it would disable word # splitting by setting IFS to empty value.) IFS=" "" $as_nl" # Find who we are. Look in the path if we contain no directory separator. as_myself= case $0 in #(( *[\\/]* ) as_myself=$0 ;; *) as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. test -r "$as_dir/$0" && as_myself=$as_dir/$0 && break done IFS=$as_save_IFS ;; esac # We did not find ourselves, most probably we were run as `sh COMMAND' # in which case we are not to be found in the path. if test "x$as_myself" = x; then as_myself=$0 fi if test ! -f "$as_myself"; then $as_echo "$as_myself: error: cannot find myself; rerun with an absolute file name" >&2 exit 1 fi # Unset variables that we do not need and which cause bugs (e.g. in # pre-3.0 UWIN ksh). But do not cause bugs in bash 2.01; the "|| exit 1" # suppresses any "Segmentation fault" message there. '((' could # trigger a bug in pdksh 5.2.14. for as_var in BASH_ENV ENV MAIL MAILPATH do eval test x\${$as_var+set} = xset \ && ( (unset $as_var) || exit 1) >/dev/null 2>&1 && unset $as_var || : done PS1='$ ' PS2='> ' PS4='+ ' # NLS nuisances. LC_ALL=C export LC_ALL LANGUAGE=C export LANGUAGE # CDPATH. (unset CDPATH) >/dev/null 2>&1 && unset CDPATH # Use a proper internal environment variable to ensure we don't fall # into an infinite loop, continuously re-executing ourselves. if test x"${_as_can_reexec}" != xno && test "x$CONFIG_SHELL" != x; then _as_can_reexec=no; export _as_can_reexec; # We cannot yet assume a decent shell, so we have to provide a # neutralization value for shells without unset; and this also # works around shells that cannot unset nonexistent variables. # Preserve -v and -x to the replacement shell. BASH_ENV=/dev/null ENV=/dev/null (unset BASH_ENV) >/dev/null 2>&1 && unset BASH_ENV ENV case $- in # (((( *v*x* | *x*v* ) as_opts=-vx ;; *v* ) as_opts=-v ;; *x* ) as_opts=-x ;; * ) as_opts= ;; esac exec $CONFIG_SHELL $as_opts "$as_myself" ${1+"$@"} # Admittedly, this is quite paranoid, since all the known shells bail # out after a failed `exec'. $as_echo "$0: could not re-execute with $CONFIG_SHELL" >&2 as_fn_exit 255 fi # We don't want this to propagate to other subprocesses. { _as_can_reexec=; unset _as_can_reexec;} if test "x$CONFIG_SHELL" = x; then as_bourne_compatible="if test -n \"\${ZSH_VERSION+set}\" && (emulate sh) >/dev/null 2>&1; then : emulate sh NULLCMD=: # Pre-4.2 versions of Zsh do word splitting on \${1+\"\$@\"}, which # is contrary to our usage. Disable this feature. alias -g '\${1+\"\$@\"}'='\"\$@\"' setopt NO_GLOB_SUBST else case \`(set -o) 2>/dev/null\` in #( *posix*) : set -o posix ;; #( *) : ;; esac fi " as_required="as_fn_return () { (exit \$1); } as_fn_success () { as_fn_return 0; } as_fn_failure () { as_fn_return 1; } as_fn_ret_success () { return 0; } as_fn_ret_failure () { return 1; } exitcode=0 as_fn_success || { exitcode=1; echo as_fn_success failed.; } as_fn_failure && { exitcode=1; echo as_fn_failure succeeded.; } as_fn_ret_success || { exitcode=1; echo as_fn_ret_success failed.; } as_fn_ret_failure && { exitcode=1; echo as_fn_ret_failure succeeded.; } if ( set x; as_fn_ret_success y && test x = \"\$1\" ); then : else exitcode=1; echo positional parameters were not saved. fi test x\$exitcode = x0 || exit 1 test -x / || exit 1" as_suggested=" as_lineno_1=";as_suggested=$as_suggested$LINENO;as_suggested=$as_suggested" as_lineno_1a=\$LINENO as_lineno_2=";as_suggested=$as_suggested$LINENO;as_suggested=$as_suggested" as_lineno_2a=\$LINENO eval 'test \"x\$as_lineno_1'\$as_run'\" != \"x\$as_lineno_2'\$as_run'\" && test \"x\`expr \$as_lineno_1'\$as_run' + 1\`\" = \"x\$as_lineno_2'\$as_run'\"' || exit 1 test -n \"\${ZSH_VERSION+set}\${BASH_VERSION+set}\" || ( ECHO='\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\' ECHO=\$ECHO\$ECHO\$ECHO\$ECHO\$ECHO ECHO=\$ECHO\$ECHO\$ECHO\$ECHO\$ECHO\$ECHO PATH=/empty FPATH=/empty; export PATH FPATH test \"X\`printf %s \$ECHO\`\" = \"X\$ECHO\" \\ || test \"X\`print -r -- \$ECHO\`\" = \"X\$ECHO\" ) || exit 1 test \$(( 1 + 1 )) = 2 || exit 1" if (eval "$as_required") 2>/dev/null; then : as_have_required=yes else as_have_required=no fi if test x$as_have_required = xyes && (eval "$as_suggested") 2>/dev/null; then : else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR as_found=false for as_dir in /bin$PATH_SEPARATOR/usr/bin$PATH_SEPARATOR$PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. as_found=: case $as_dir in #( /*) for as_base in sh bash ksh sh5; do # Try only shells that exist, to save several forks. as_shell=$as_dir/$as_base if { test -f "$as_shell" || test -f "$as_shell.exe"; } && { $as_echo "$as_bourne_compatible""$as_required" | as_run=a "$as_shell"; } 2>/dev/null; then : CONFIG_SHELL=$as_shell as_have_required=yes if { $as_echo "$as_bourne_compatible""$as_suggested" | as_run=a "$as_shell"; } 2>/dev/null; then : break 2 fi fi done;; esac as_found=false done $as_found || { if { test -f "$SHELL" || test -f "$SHELL.exe"; } && { $as_echo "$as_bourne_compatible""$as_required" | as_run=a "$SHELL"; } 2>/dev/null; then : CONFIG_SHELL=$SHELL as_have_required=yes fi; } IFS=$as_save_IFS if test "x$CONFIG_SHELL" != x; then : export CONFIG_SHELL # We cannot yet assume a decent shell, so we have to provide a # neutralization value for shells without unset; and this also # works around shells that cannot unset nonexistent variables. # Preserve -v and -x to the replacement shell. BASH_ENV=/dev/null ENV=/dev/null (unset BASH_ENV) >/dev/null 2>&1 && unset BASH_ENV ENV case $- in # (((( *v*x* | *x*v* ) as_opts=-vx ;; *v* ) as_opts=-v ;; *x* ) as_opts=-x ;; * ) as_opts= ;; esac exec $CONFIG_SHELL $as_opts "$as_myself" ${1+"$@"} # Admittedly, this is quite paranoid, since all the known shells bail # out after a failed `exec'. $as_echo "$0: could not re-execute with $CONFIG_SHELL" >&2 exit 255 fi if test x$as_have_required = xno; then : $as_echo "$0: This script requires a shell more modern than all" $as_echo "$0: the shells that I found on your system." if test x${ZSH_VERSION+set} = xset ; then $as_echo "$0: In particular, zsh $ZSH_VERSION has bugs and should" $as_echo "$0: be upgraded to zsh 4.3.4 or later." else $as_echo "$0: Please tell bug-autoconf@gnu.org and jl@hunch.net about $0: your system, including any error possibly output before $0: this message. Then install a modern shell, or manually $0: run the script under such a shell if you do have one." fi exit 1 fi fi fi SHELL=${CONFIG_SHELL-/bin/sh} export SHELL # Unset more variables known to interfere with behavior of common tools. CLICOLOR_FORCE= GREP_OPTIONS= unset CLICOLOR_FORCE GREP_OPTIONS ## --------------------- ## ## M4sh Shell Functions. ## ## --------------------- ## # as_fn_unset VAR # --------------- # Portably unset VAR. as_fn_unset () { { eval $1=; unset $1;} } as_unset=as_fn_unset # as_fn_set_status STATUS # ----------------------- # Set $? to STATUS, without forking. as_fn_set_status () { return $1 } # as_fn_set_status # as_fn_exit STATUS # ----------------- # Exit the shell with STATUS, even in a "trap 0" or "set -e" context. as_fn_exit () { set +e as_fn_set_status $1 exit $1 } # as_fn_exit # as_fn_mkdir_p # ------------- # Create "$as_dir" as a directory, including parents if necessary. as_fn_mkdir_p () { case $as_dir in #( -*) as_dir=./$as_dir;; esac test -d "$as_dir" || eval $as_mkdir_p || { as_dirs= while :; do case $as_dir in #( *\'*) as_qdir=`$as_echo "$as_dir" | sed "s/'/'\\\\\\\\''/g"`;; #'( *) as_qdir=$as_dir;; esac as_dirs="'$as_qdir' $as_dirs" as_dir=`$as_dirname -- "$as_dir" || $as_expr X"$as_dir" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ X"$as_dir" : 'X\(//\)[^/]' \| \ X"$as_dir" : 'X\(//\)$' \| \ X"$as_dir" : 'X\(/\)' \| . 2>/dev/null || $as_echo X"$as_dir" | sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ s//\1/ q } /^X\(\/\/\)[^/].*/{ s//\1/ q } /^X\(\/\/\)$/{ s//\1/ q } /^X\(\/\).*/{ s//\1/ q } s/.*/./; q'` test -d "$as_dir" && break done test -z "$as_dirs" || eval "mkdir $as_dirs" } || test -d "$as_dir" || as_fn_error $? "cannot create directory $as_dir" } # as_fn_mkdir_p # as_fn_executable_p FILE # ----------------------- # Test if FILE is an executable regular file. as_fn_executable_p () { test -f "$1" && test -x "$1" } # as_fn_executable_p # as_fn_append VAR VALUE # ---------------------- # Append the text in VALUE to the end of the definition contained in VAR. Take # advantage of any shell optimizations that allow amortized linear growth over # repeated appends, instead of the typical quadratic growth present in naive # implementations. if (eval "as_var=1; as_var+=2; test x\$as_var = x12") 2>/dev/null; then : eval 'as_fn_append () { eval $1+=\$2 }' else as_fn_append () { eval $1=\$$1\$2 } fi # as_fn_append # as_fn_arith ARG... # ------------------ # Perform arithmetic evaluation on the ARGs, and store the result in the # global $as_val. Take advantage of shells that can avoid forks. The arguments # must be portable across $(()) and expr. if (eval "test \$(( 1 + 1 )) = 2") 2>/dev/null; then : eval 'as_fn_arith () { as_val=$(( $* )) }' else as_fn_arith () { as_val=`expr "$@" || test $? -eq 1` } fi # as_fn_arith # as_fn_error STATUS ERROR [LINENO LOG_FD] # ---------------------------------------- # Output "`basename $0`: error: ERROR" to stderr. If LINENO and LOG_FD are # provided, also output the error to LOG_FD, referencing LINENO. Then exit the # script with STATUS, using 1 if that was 0. as_fn_error () { as_status=$1; test $as_status -eq 0 && as_status=1 if test "$4"; then as_lineno=${as_lineno-"$3"} as_lineno_stack=as_lineno_stack=$as_lineno_stack $as_echo "$as_me:${as_lineno-$LINENO}: error: $2" >&$4 fi $as_echo "$as_me: error: $2" >&2 as_fn_exit $as_status } # as_fn_error if expr a : '\(a\)' >/dev/null 2>&1 && test "X`expr 00001 : '.*\(...\)'`" = X001; then as_expr=expr else as_expr=false fi if (basename -- /) >/dev/null 2>&1 && test "X`basename -- / 2>&1`" = "X/"; then as_basename=basename else as_basename=false fi if (as_dir=`dirname -- /` && test "X$as_dir" = X/) >/dev/null 2>&1; then as_dirname=dirname else as_dirname=false fi as_me=`$as_basename -- "$0" || $as_expr X/"$0" : '.*/\([^/][^/]*\)/*$' \| \ X"$0" : 'X\(//\)$' \| \ X"$0" : 'X\(/\)' \| . 2>/dev/null || $as_echo X/"$0" | sed '/^.*\/\([^/][^/]*\)\/*$/{ s//\1/ q } /^X\/\(\/\/\)$/{ s//\1/ q } /^X\/\(\/\).*/{ s//\1/ q } s/.*/./; q'` # Avoid depending upon Character Ranges. as_cr_letters='abcdefghijklmnopqrstuvwxyz' as_cr_LETTERS='ABCDEFGHIJKLMNOPQRSTUVWXYZ' as_cr_Letters=$as_cr_letters$as_cr_LETTERS as_cr_digits='0123456789' as_cr_alnum=$as_cr_Letters$as_cr_digits as_lineno_1=$LINENO as_lineno_1a=$LINENO as_lineno_2=$LINENO as_lineno_2a=$LINENO eval 'test "x$as_lineno_1'$as_run'" != "x$as_lineno_2'$as_run'" && test "x`expr $as_lineno_1'$as_run' + 1`" = "x$as_lineno_2'$as_run'"' || { # Blame Lee E. McMahon (1931-1989) for sed's syntax. :-) sed -n ' p /[$]LINENO/= ' <$as_myself | sed ' s/[$]LINENO.*/&-/ t lineno b :lineno N :loop s/[$]LINENO\([^'$as_cr_alnum'_].*\n\)\(.*\)/\2\1\2/ t loop s/-\n.*// ' >$as_me.lineno && chmod +x "$as_me.lineno" || { $as_echo "$as_me: error: cannot create $as_me.lineno; rerun with a POSIX shell" >&2; as_fn_exit 1; } # If we had to re-execute with $CONFIG_SHELL, we're ensured to have # already done that, so ensure we don't try to do so again and fall # in an infinite loop. This has already happened in practice. _as_can_reexec=no; export _as_can_reexec # Don't try to exec as it changes $[0], causing all sort of problems # (the dirname of $[0] is not the place where we might find the # original and so on. Autoconf is especially sensitive to this). . "./$as_me.lineno" # Exit status is that of the last command. exit } ECHO_C= ECHO_N= ECHO_T= case `echo -n x` in #((((( -n*) case `echo 'xy\c'` in *c*) ECHO_T=' ';; # ECHO_T is single tab character. xy) ECHO_C='\c';; *) echo `echo ksh88 bug on AIX 6.1` > /dev/null ECHO_T=' ';; esac;; *) ECHO_N='-n';; esac rm -f conf$$ conf$$.exe conf$$.file if test -d conf$$.dir; then rm -f conf$$.dir/conf$$.file else rm -f conf$$.dir mkdir conf$$.dir 2>/dev/null fi if (echo >conf$$.file) 2>/dev/null; then if ln -s conf$$.file conf$$ 2>/dev/null; then as_ln_s='ln -s' # ... but there are two gotchas: # 1) On MSYS, both `ln -s file dir' and `ln file dir' fail. # 2) DJGPP < 2.04 has no symlinks; `ln -s' creates a wrapper executable. # In both cases, we have to default to `cp -pR'. ln -s conf$$.file conf$$.dir 2>/dev/null && test ! -f conf$$.exe || as_ln_s='cp -pR' elif ln conf$$.file conf$$ 2>/dev/null; then as_ln_s=ln else as_ln_s='cp -pR' fi else as_ln_s='cp -pR' fi rm -f conf$$ conf$$.exe conf$$.dir/conf$$.file conf$$.file rmdir conf$$.dir 2>/dev/null if mkdir -p . 2>/dev/null; then as_mkdir_p='mkdir -p "$as_dir"' else test -d ./-p && rmdir ./-p as_mkdir_p=false fi as_test_x='test -x' as_executable_p=as_fn_executable_p # Sed expression to map a string onto a valid CPP name. as_tr_cpp="eval sed 'y%*$as_cr_letters%P$as_cr_LETTERS%;s%[^_$as_cr_alnum]%_%g'" # Sed expression to map a string onto a valid variable name. as_tr_sh="eval sed 'y%*+%pp%;s%[^_$as_cr_alnum]%_%g'" SHELL=${CONFIG_SHELL-/bin/sh} test -n "$DJDIR" || exec 7<&0 &1 # Name of the host. # hostname on some systems (SVR3.2, old GNU/Linux) returns a bogus exit status, # so uname gets run too. ac_hostname=`(hostname || uname -n) 2>/dev/null | sed 1q` # # Initializations. # ac_default_prefix=/usr/local ac_clean_files= ac_config_libobj_dir=. LIBOBJS= cross_compiling=no subdirs= MFLAGS= MAKEFLAGS= # Identity of this package. PACKAGE_NAME='vowpal_wabbit' PACKAGE_TARNAME='vowpal_wabbit' PACKAGE_VERSION='8.5.0' PACKAGE_STRING='vowpal_wabbit 8.5.0' PACKAGE_BUGREPORT='jl@hunch.net' PACKAGE_URL='https://github.com/JohnLangford/vowpal_wabbit' # Factoring default headers for most tests. ac_includes_default="\ #include #ifdef HAVE_SYS_TYPES_H # include #endif #ifdef HAVE_SYS_STAT_H # include #endif #ifdef STDC_HEADERS # include # include #else # ifdef HAVE_STDLIB_H # include # endif #endif #ifdef HAVE_STRING_H # if !defined STDC_HEADERS && defined HAVE_MEMORY_H # include # endif # include #endif #ifdef HAVE_STRINGS_H # include #endif #ifdef HAVE_INTTYPES_H # include #endif #ifdef HAVE_STDINT_H # include #endif #ifdef HAVE_UNISTD_H # include #endif" ac_subst_vars='am__EXEEXT_FALSE am__EXEEXT_TRUE LTLIBOBJS LIBOBJS FREEBSD_FALSE FREEBSD_TRUE CLANG_LIBCXX_FALSE CLANG_LIBCXX_TRUE VWBUG_FALSE VWBUG_TRUE PROFILE_FALSE PROFILE_TRUE PARALLELIZE_FALSE PARALLELIZE_TRUE NITPICK_FALSE NITPICK_TRUE PTHREAD_CFLAGS PTHREAD_LIBS PTHREAD_CC ax_pthread_config ZLIB_LDFLAGS ZLIB_CPPFLAGS BOOST_PROGRAM_OPTIONS_LIB BOOST_LDFLAGS BOOST_CPPFLAGS HAVE_CXX11 CPP LT_SYS_LIBRARY_PATH OTOOL64 OTOOL LIPO NMEDIT DSYMUTIL MANIFEST_TOOL RANLIB ac_ct_AR AR DLLTOOL OBJDUMP LN_S NM ac_ct_DUMPBIN DUMPBIN LD FGREP EGREP GREP SED am__fastdepCC_FALSE am__fastdepCC_TRUE CCDEPMODE ac_ct_CC CFLAGS CC LIBTOOL host_os host_vendor host_cpu host build_os build_vendor build_cpu build CXXCPP am__fastdepCXX_FALSE am__fastdepCXX_TRUE CXXDEPMODE am__nodep AMDEPBACKSLASH AMDEP_FALSE AMDEP_TRUE am__quote am__include DEPDIR OBJEXT EXEEXT ac_ct_CXX CPPFLAGS LDFLAGS CXXFLAGS CXX AM_BACKSLASH AM_DEFAULT_VERBOSITY AM_DEFAULT_V AM_V am__untar am__tar AMTAR am__leading_dot SET_MAKE AWK mkdir_p MKDIR_P INSTALL_STRIP_PROGRAM STRIP install_sh MAKEINFO AUTOHEADER AUTOMAKE AUTOCONF ACLOCAL VERSION PACKAGE CYGPATH_W am__isrc INSTALL_DATA INSTALL_SCRIPT INSTALL_PROGRAM target_alias host_alias build_alias LIBS ECHO_T ECHO_N ECHO_C DEFS mandir localedir libdir psdir pdfdir dvidir htmldir infodir docdir oldincludedir includedir runstatedir localstatedir sharedstatedir sysconfdir datadir datarootdir libexecdir sbindir bindir program_transform_name prefix exec_prefix PACKAGE_URL PACKAGE_BUGREPORT PACKAGE_STRING PACKAGE_VERSION PACKAGE_TARNAME PACKAGE_NAME PATH_SEPARATOR SHELL' ac_subst_files='' ac_user_opts=' enable_option_checking enable_silent_rules enable_dependency_tracking enable_shared enable_static with_pic enable_fast_install with_aix_soname with_gnu_ld with_sysroot enable_libtool_lock with_boost with_boost_libdir with_boost_program_options with_zlib enable_nitpicking enable_parallelization enable_profiling enable_debug enable_libc__ ' ac_precious_vars='build_alias host_alias target_alias CXX CXXFLAGS LDFLAGS LIBS CPPFLAGS CCC CXXCPP CC CFLAGS LT_SYS_LIBRARY_PATH CPP' # Initialize some variables set by options. ac_init_help= ac_init_version=false ac_unrecognized_opts= ac_unrecognized_sep= # The variables have the same names as the options, with # dashes changed to underlines. cache_file=/dev/null exec_prefix=NONE no_create= no_recursion= prefix=NONE program_prefix=NONE program_suffix=NONE program_transform_name=s,x,x, silent= site= srcdir= verbose= x_includes=NONE x_libraries=NONE # Installation directory options. # These are left unexpanded so users can "make install exec_prefix=/foo" # and all the variables that are supposed to be based on exec_prefix # by default will actually change. # Use braces instead of parens because sh, perl, etc. also accept them. # (The list follows the same order as the GNU Coding Standards.) bindir='${exec_prefix}/bin' sbindir='${exec_prefix}/sbin' libexecdir='${exec_prefix}/libexec' datarootdir='${prefix}/share' datadir='${datarootdir}' sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' infodir='${datarootdir}/info' htmldir='${docdir}' dvidir='${docdir}' pdfdir='${docdir}' psdir='${docdir}' libdir='${exec_prefix}/lib' localedir='${datarootdir}/locale' mandir='${datarootdir}/man' ac_prev= ac_dashdash= for ac_option do # If the previous option needs an argument, assign it. if test -n "$ac_prev"; then eval $ac_prev=\$ac_option ac_prev= continue fi case $ac_option in *=?*) ac_optarg=`expr "X$ac_option" : '[^=]*=\(.*\)'` ;; *=) ac_optarg= ;; *) ac_optarg=yes ;; esac # Accept the important Cygnus configure options, so we can diagnose typos. case $ac_dashdash$ac_option in --) ac_dashdash=yes ;; -bindir | --bindir | --bindi | --bind | --bin | --bi) ac_prev=bindir ;; -bindir=* | --bindir=* | --bindi=* | --bind=* | --bin=* | --bi=*) bindir=$ac_optarg ;; -build | --build | --buil | --bui | --bu) ac_prev=build_alias ;; -build=* | --build=* | --buil=* | --bui=* | --bu=*) build_alias=$ac_optarg ;; -cache-file | --cache-file | --cache-fil | --cache-fi \ | --cache-f | --cache- | --cache | --cach | --cac | --ca | --c) ac_prev=cache_file ;; -cache-file=* | --cache-file=* | --cache-fil=* | --cache-fi=* \ | --cache-f=* | --cache-=* | --cache=* | --cach=* | --cac=* | --ca=* | --c=*) cache_file=$ac_optarg ;; --config-cache | -C) cache_file=config.cache ;; -datadir | --datadir | --datadi | --datad) ac_prev=datadir ;; -datadir=* | --datadir=* | --datadi=* | --datad=*) datadir=$ac_optarg ;; -datarootdir | --datarootdir | --datarootdi | --datarootd | --dataroot \ | --dataroo | --dataro | --datar) ac_prev=datarootdir ;; -datarootdir=* | --datarootdir=* | --datarootdi=* | --datarootd=* \ | --dataroot=* | --dataroo=* | --dataro=* | --datar=*) datarootdir=$ac_optarg ;; -disable-* | --disable-*) ac_useropt=`expr "x$ac_option" : 'x-*disable-\(.*\)'` # Reject names that are not valid shell variable names. expr "x$ac_useropt" : ".*[^-+._$as_cr_alnum]" >/dev/null && as_fn_error $? "invalid feature name: $ac_useropt" ac_useropt_orig=$ac_useropt ac_useropt=`$as_echo "$ac_useropt" | sed 's/[-+.]/_/g'` case $ac_user_opts in *" "enable_$ac_useropt" "*) ;; *) ac_unrecognized_opts="$ac_unrecognized_opts$ac_unrecognized_sep--disable-$ac_useropt_orig" ac_unrecognized_sep=', ';; esac eval enable_$ac_useropt=no ;; -docdir | --docdir | --docdi | --doc | --do) ac_prev=docdir ;; -docdir=* | --docdir=* | --docdi=* | --doc=* | --do=*) docdir=$ac_optarg ;; -dvidir | --dvidir | --dvidi | --dvid | --dvi | --dv) ac_prev=dvidir ;; -dvidir=* | --dvidir=* | --dvidi=* | --dvid=* | --dvi=* | --dv=*) dvidir=$ac_optarg ;; -enable-* | --enable-*) ac_useropt=`expr "x$ac_option" : 'x-*enable-\([^=]*\)'` # Reject names that are not valid shell variable names. expr "x$ac_useropt" : ".*[^-+._$as_cr_alnum]" >/dev/null && as_fn_error $? "invalid feature name: $ac_useropt" ac_useropt_orig=$ac_useropt ac_useropt=`$as_echo "$ac_useropt" | sed 's/[-+.]/_/g'` case $ac_user_opts in *" "enable_$ac_useropt" "*) ;; *) ac_unrecognized_opts="$ac_unrecognized_opts$ac_unrecognized_sep--enable-$ac_useropt_orig" ac_unrecognized_sep=', ';; esac eval enable_$ac_useropt=\$ac_optarg ;; -exec-prefix | --exec_prefix | --exec-prefix | --exec-prefi \ | --exec-pref | --exec-pre | --exec-pr | --exec-p | --exec- \ | --exec | --exe | --ex) ac_prev=exec_prefix ;; -exec-prefix=* | --exec_prefix=* | --exec-prefix=* | --exec-prefi=* \ | --exec-pref=* | --exec-pre=* | --exec-pr=* | --exec-p=* | --exec-=* \ | --exec=* | --exe=* | --ex=*) exec_prefix=$ac_optarg ;; -gas | --gas | --ga | --g) # Obsolete; use --with-gas. with_gas=yes ;; -help | --help | --hel | --he | -h) ac_init_help=long ;; -help=r* | --help=r* | --hel=r* | --he=r* | -hr*) ac_init_help=recursive ;; -help=s* | --help=s* | --hel=s* | --he=s* | -hs*) ac_init_help=short ;; -host | --host | --hos | --ho) ac_prev=host_alias ;; -host=* | --host=* | --hos=* | --ho=*) host_alias=$ac_optarg ;; -htmldir | --htmldir | --htmldi | --htmld | --html | --htm | --ht) ac_prev=htmldir ;; -htmldir=* | --htmldir=* | --htmldi=* | --htmld=* | --html=* | --htm=* \ | --ht=*) htmldir=$ac_optarg ;; -includedir | --includedir | --includedi | --included | --include \ | --includ | --inclu | --incl | --inc) ac_prev=includedir ;; -includedir=* | --includedir=* | --includedi=* | --included=* | --include=* \ | --includ=* | --inclu=* | --incl=* | --inc=*) includedir=$ac_optarg ;; -infodir | --infodir | --infodi | --infod | --info | --inf) ac_prev=infodir ;; -infodir=* | --infodir=* | --infodi=* | --infod=* | --info=* | --inf=*) infodir=$ac_optarg ;; -libdir | --libdir | --libdi | --libd) ac_prev=libdir ;; -libdir=* | --libdir=* | --libdi=* | --libd=*) libdir=$ac_optarg ;; -libexecdir | --libexecdir | --libexecdi | --libexecd | --libexec \ | --libexe | --libex | --libe) ac_prev=libexecdir ;; -libexecdir=* | --libexecdir=* | --libexecdi=* | --libexecd=* | --libexec=* \ | --libexe=* | --libex=* | --libe=*) libexecdir=$ac_optarg ;; -localedir | --localedir | --localedi | --localed | --locale) ac_prev=localedir ;; -localedir=* | --localedir=* | --localedi=* | --localed=* | --locale=*) localedir=$ac_optarg ;; -localstatedir | --localstatedir | --localstatedi | --localstated \ | --localstate | --localstat | --localsta | --localst | --locals) ac_prev=localstatedir ;; -localstatedir=* | --localstatedir=* | --localstatedi=* | --localstated=* \ | --localstate=* | --localstat=* | --localsta=* | --localst=* | --locals=*) localstatedir=$ac_optarg ;; -mandir | --mandir | --mandi | --mand | --man | --ma | --m) ac_prev=mandir ;; -mandir=* | --mandir=* | --mandi=* | --mand=* | --man=* | --ma=* | --m=*) mandir=$ac_optarg ;; -nfp | --nfp | --nf) # Obsolete; use --without-fp. with_fp=no ;; -no-create | --no-create | --no-creat | --no-crea | --no-cre \ | --no-cr | --no-c | -n) no_create=yes ;; -no-recursion | --no-recursion | --no-recursio | --no-recursi \ | --no-recurs | --no-recur | --no-recu | --no-rec | --no-re | --no-r) no_recursion=yes ;; -oldincludedir | --oldincludedir | --oldincludedi | --oldincluded \ | --oldinclude | --oldinclud | --oldinclu | --oldincl | --oldinc \ | --oldin | --oldi | --old | --ol | --o) ac_prev=oldincludedir ;; -oldincludedir=* | --oldincludedir=* | --oldincludedi=* | --oldincluded=* \ | --oldinclude=* | --oldinclud=* | --oldinclu=* | --oldincl=* | --oldinc=* \ | --oldin=* | --oldi=* | --old=* | --ol=* | --o=*) oldincludedir=$ac_optarg ;; -prefix | --prefix | --prefi | --pref | --pre | --pr | --p) ac_prev=prefix ;; -prefix=* | --prefix=* | --prefi=* | --pref=* | --pre=* | --pr=* | --p=*) prefix=$ac_optarg ;; -program-prefix | --program-prefix | --program-prefi | --program-pref \ | --program-pre | --program-pr | --program-p) ac_prev=program_prefix ;; -program-prefix=* | --program-prefix=* | --program-prefi=* \ | --program-pref=* | --program-pre=* | --program-pr=* | --program-p=*) program_prefix=$ac_optarg ;; -program-suffix | --program-suffix | --program-suffi | --program-suff \ | --program-suf | --program-su | --program-s) ac_prev=program_suffix ;; -program-suffix=* | --program-suffix=* | --program-suffi=* \ | --program-suff=* | --program-suf=* | --program-su=* | --program-s=*) program_suffix=$ac_optarg ;; -program-transform-name | --program-transform-name \ | --program-transform-nam | --program-transform-na \ | --program-transform-n | --program-transform- \ | --program-transform | --program-transfor \ | --program-transfo | --program-transf \ | --program-trans | --program-tran \ | --progr-tra | --program-tr | --program-t) ac_prev=program_transform_name ;; -program-transform-name=* | --program-transform-name=* \ | --program-transform-nam=* | --program-transform-na=* \ | --program-transform-n=* | --program-transform-=* \ | --program-transform=* | --program-transfor=* \ | --program-transfo=* | --program-transf=* \ | --program-trans=* | --program-tran=* \ | --progr-tra=* | --program-tr=* | --program-t=*) program_transform_name=$ac_optarg ;; -pdfdir | --pdfdir | --pdfdi | --pdfd | --pdf | --pd) ac_prev=pdfdir ;; -pdfdir=* | --pdfdir=* | --pdfdi=* | --pdfd=* | --pdf=* | --pd=*) pdfdir=$ac_optarg ;; -psdir | --psdir | --psdi | --psd | --ps) ac_prev=psdir ;; -psdir=* | --psdir=* | --psdi=* | --psd=* | --ps=*) psdir=$ac_optarg ;; -q | -quiet | --quiet | --quie | --qui | --qu | --q \ | -silent | --silent | --silen | --sile | --sil) silent=yes ;; -runstatedir | --runstatedir | --runstatedi | --runstated \ | --runstate | --runstat | --runsta | --runst | --runs \ | --run | --ru | --r) ac_prev=runstatedir ;; -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ | --run=* | --ru=* | --r=*) runstatedir=$ac_optarg ;; -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ | --sbi=* | --sb=*) sbindir=$ac_optarg ;; -sharedstatedir | --sharedstatedir | --sharedstatedi \ | --sharedstated | --sharedstate | --sharedstat | --sharedsta \ | --sharedst | --shareds | --shared | --share | --shar \ | --sha | --sh) ac_prev=sharedstatedir ;; -sharedstatedir=* | --sharedstatedir=* | --sharedstatedi=* \ | --sharedstated=* | --sharedstate=* | --sharedstat=* | --sharedsta=* \ | --sharedst=* | --shareds=* | --shared=* | --share=* | --shar=* \ | --sha=* | --sh=*) sharedstatedir=$ac_optarg ;; -site | --site | --sit) ac_prev=site ;; -site=* | --site=* | --sit=*) site=$ac_optarg ;; -srcdir | --srcdir | --srcdi | --srcd | --src | --sr) ac_prev=srcdir ;; -srcdir=* | --srcdir=* | --srcdi=* | --srcd=* | --src=* | --sr=*) srcdir=$ac_optarg ;; -sysconfdir | --sysconfdir | --sysconfdi | --sysconfd | --sysconf \ | --syscon | --sysco | --sysc | --sys | --sy) ac_prev=sysconfdir ;; -sysconfdir=* | --sysconfdir=* | --sysconfdi=* | --sysconfd=* | --sysconf=* \ | --syscon=* | --sysco=* | --sysc=* | --sys=* | --sy=*) sysconfdir=$ac_optarg ;; -target | --target | --targe | --targ | --tar | --ta | --t) ac_prev=target_alias ;; -target=* | --target=* | --targe=* | --targ=* | --tar=* | --ta=* | --t=*) target_alias=$ac_optarg ;; -v | -verbose | --verbose | --verbos | --verbo | --verb) verbose=yes ;; -version | --version | --versio | --versi | --vers | -V) ac_init_version=: ;; -with-* | --with-*) ac_useropt=`expr "x$ac_option" : 'x-*with-\([^=]*\)'` # Reject names that are not valid shell variable names. expr "x$ac_useropt" : ".*[^-+._$as_cr_alnum]" >/dev/null && as_fn_error $? "invalid package name: $ac_useropt" ac_useropt_orig=$ac_useropt ac_useropt=`$as_echo "$ac_useropt" | sed 's/[-+.]/_/g'` case $ac_user_opts in *" "with_$ac_useropt" "*) ;; *) ac_unrecognized_opts="$ac_unrecognized_opts$ac_unrecognized_sep--with-$ac_useropt_orig" ac_unrecognized_sep=', ';; esac eval with_$ac_useropt=\$ac_optarg ;; -without-* | --without-*) ac_useropt=`expr "x$ac_option" : 'x-*without-\(.*\)'` # Reject names that are not valid shell variable names. expr "x$ac_useropt" : ".*[^-+._$as_cr_alnum]" >/dev/null && as_fn_error $? "invalid package name: $ac_useropt" ac_useropt_orig=$ac_useropt ac_useropt=`$as_echo "$ac_useropt" | sed 's/[-+.]/_/g'` case $ac_user_opts in *" "with_$ac_useropt" "*) ;; *) ac_unrecognized_opts="$ac_unrecognized_opts$ac_unrecognized_sep--without-$ac_useropt_orig" ac_unrecognized_sep=', ';; esac eval with_$ac_useropt=no ;; --x) # Obsolete; use --with-x. with_x=yes ;; -x-includes | --x-includes | --x-include | --x-includ | --x-inclu \ | --x-incl | --x-inc | --x-in | --x-i) ac_prev=x_includes ;; -x-includes=* | --x-includes=* | --x-include=* | --x-includ=* | --x-inclu=* \ | --x-incl=* | --x-inc=* | --x-in=* | --x-i=*) x_includes=$ac_optarg ;; -x-libraries | --x-libraries | --x-librarie | --x-librari \ | --x-librar | --x-libra | --x-libr | --x-lib | --x-li | --x-l) ac_prev=x_libraries ;; -x-libraries=* | --x-libraries=* | --x-librarie=* | --x-librari=* \ | --x-librar=* | --x-libra=* | --x-libr=* | --x-lib=* | --x-li=* | --x-l=*) x_libraries=$ac_optarg ;; -*) as_fn_error $? "unrecognized option: \`$ac_option' Try \`$0 --help' for more information" ;; *=*) ac_envvar=`expr "x$ac_option" : 'x\([^=]*\)='` # Reject names that are not valid shell variable names. case $ac_envvar in #( '' | [0-9]* | *[!_$as_cr_alnum]* ) as_fn_error $? "invalid variable name: \`$ac_envvar'" ;; esac eval $ac_envvar=\$ac_optarg export $ac_envvar ;; *) # FIXME: should be removed in autoconf 3.0. $as_echo "$as_me: WARNING: you should use --build, --host, --target" >&2 expr "x$ac_option" : ".*[^-._$as_cr_alnum]" >/dev/null && $as_echo "$as_me: WARNING: invalid host type: $ac_option" >&2 : "${build_alias=$ac_option} ${host_alias=$ac_option} ${target_alias=$ac_option}" ;; esac done if test -n "$ac_prev"; then ac_option=--`echo $ac_prev | sed 's/_/-/g'` as_fn_error $? "missing argument to $ac_option" fi if test -n "$ac_unrecognized_opts"; then case $enable_option_checking in no) ;; fatal) as_fn_error $? "unrecognized options: $ac_unrecognized_opts" ;; *) $as_echo "$as_me: WARNING: unrecognized options: $ac_unrecognized_opts" >&2 ;; esac fi # Check all directory arguments for consistency. for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ libdir localedir mandir runstatedir do eval ac_val=\$$ac_var # Remove trailing slashes. case $ac_val in */ ) ac_val=`expr "X$ac_val" : 'X\(.*[^/]\)' \| "X$ac_val" : 'X\(.*\)'` eval $ac_var=\$ac_val;; esac # Be sure to have absolute directory names. case $ac_val in [\\/$]* | ?:[\\/]* ) continue;; NONE | '' ) case $ac_var in *prefix ) continue;; esac;; esac as_fn_error $? "expected an absolute directory name for --$ac_var: $ac_val" done # There might be people who depend on the old broken behavior: `$host' # used to hold the argument of --host etc. # FIXME: To remove some day. build=$build_alias host=$host_alias target=$target_alias # FIXME: To remove some day. if test "x$host_alias" != x; then if test "x$build_alias" = x; then cross_compiling=maybe elif test "x$build_alias" != "x$host_alias"; then cross_compiling=yes fi fi ac_tool_prefix= test -n "$host_alias" && ac_tool_prefix=$host_alias- test "$silent" = yes && exec 6>/dev/null ac_pwd=`pwd` && test -n "$ac_pwd" && ac_ls_di=`ls -di .` && ac_pwd_ls_di=`cd "$ac_pwd" && ls -di .` || as_fn_error $? "working directory cannot be determined" test "X$ac_ls_di" = "X$ac_pwd_ls_di" || as_fn_error $? "pwd does not report name of working directory" # Find the source files, if location was not specified. if test -z "$srcdir"; then ac_srcdir_defaulted=yes # Try the directory containing this script, then the parent directory. ac_confdir=`$as_dirname -- "$as_myself" || $as_expr X"$as_myself" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ X"$as_myself" : 'X\(//\)[^/]' \| \ X"$as_myself" : 'X\(//\)$' \| \ X"$as_myself" : 'X\(/\)' \| . 2>/dev/null || $as_echo X"$as_myself" | sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ s//\1/ q } /^X\(\/\/\)[^/].*/{ s//\1/ q } /^X\(\/\/\)$/{ s//\1/ q } /^X\(\/\).*/{ s//\1/ q } s/.*/./; q'` srcdir=$ac_confdir if test ! -r "$srcdir/$ac_unique_file"; then srcdir=.. fi else ac_srcdir_defaulted=no fi if test ! -r "$srcdir/$ac_unique_file"; then test "$ac_srcdir_defaulted" = yes && srcdir="$ac_confdir or .." as_fn_error $? "cannot find sources ($ac_unique_file) in $srcdir" fi ac_msg="sources are in $srcdir, but \`cd $srcdir' does not work" ac_abs_confdir=`( cd "$srcdir" && test -r "./$ac_unique_file" || as_fn_error $? "$ac_msg" pwd)` # When building in place, set srcdir=. if test "$ac_abs_confdir" = "$ac_pwd"; then srcdir=. fi # Remove unnecessary trailing slashes from srcdir. # Double slashes in file names in object file debugging info # mess up M-x gdb in Emacs. case $srcdir in */) srcdir=`expr "X$srcdir" : 'X\(.*[^/]\)' \| "X$srcdir" : 'X\(.*\)'`;; esac for ac_var in $ac_precious_vars; do eval ac_env_${ac_var}_set=\${${ac_var}+set} eval ac_env_${ac_var}_value=\$${ac_var} eval ac_cv_env_${ac_var}_set=\${${ac_var}+set} eval ac_cv_env_${ac_var}_value=\$${ac_var} done # # Report the --help message. # if test "$ac_init_help" = "long"; then # Omit some internal or obsolete options to make the list less imposing. # This message is too long to be a string in the A/UX 3.1 sh. cat <<_ACEOF \`configure' configures vowpal_wabbit 8.5.0 to adapt to many kinds of systems. Usage: $0 [OPTION]... [VAR=VALUE]... To assign environment variables (e.g., CC, CFLAGS...), specify them as VAR=VALUE. See below for descriptions of some of the useful variables. Defaults for the options are specified in brackets. Configuration: -h, --help display this help and exit --help=short display options specific to this package --help=recursive display the short help of all the included packages -V, --version display version information and exit -q, --quiet, --silent do not print \`checking ...' messages --cache-file=FILE cache test results in FILE [disabled] -C, --config-cache alias for \`--cache-file=config.cache' -n, --no-create do not create output files --srcdir=DIR find the sources in DIR [configure dir or \`..'] Installation directories: --prefix=PREFIX install architecture-independent files in PREFIX [$ac_default_prefix] --exec-prefix=EPREFIX install architecture-dependent files in EPREFIX [PREFIX] By default, \`make install' will install all the files in \`$ac_default_prefix/bin', \`$ac_default_prefix/lib' etc. You can specify an installation prefix other than \`$ac_default_prefix' using \`--prefix', for instance \`--prefix=\$HOME'. For better control, use the options below. Fine tuning of the installation directories: --bindir=DIR user executables [EPREFIX/bin] --sbindir=DIR system admin executables [EPREFIX/sbin] --libexecdir=DIR program executables [EPREFIX/libexec] --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] --datarootdir=DIR read-only arch.-independent data root [PREFIX/share] --datadir=DIR read-only architecture-independent data [DATAROOTDIR] --infodir=DIR info documentation [DATAROOTDIR/info] --localedir=DIR locale-dependent data [DATAROOTDIR/locale] --mandir=DIR man documentation [DATAROOTDIR/man] --docdir=DIR documentation root [DATAROOTDIR/doc/vowpal_wabbit] --htmldir=DIR html documentation [DOCDIR] --dvidir=DIR dvi documentation [DOCDIR] --pdfdir=DIR pdf documentation [DOCDIR] --psdir=DIR ps documentation [DOCDIR] _ACEOF cat <<\_ACEOF Program names: --program-prefix=PREFIX prepend PREFIX to installed program names --program-suffix=SUFFIX append SUFFIX to installed program names --program-transform-name=PROGRAM run sed PROGRAM on installed program names System types: --build=BUILD configure for building on BUILD [guessed] --host=HOST cross-compile to build programs to run on HOST [BUILD] _ACEOF fi if test -n "$ac_init_help"; then case $ac_init_help in short | recursive ) echo "Configuration of vowpal_wabbit 8.5.0:";; esac cat <<\_ACEOF Optional Features: --disable-option-checking ignore unrecognized --enable/--with options --disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no) --enable-FEATURE[=ARG] include FEATURE [ARG=yes] --enable-silent-rules less verbose build output (undo: "make V=1") --disable-silent-rules verbose build output (undo: "make V=0") --enable-dependency-tracking do not reject slow dependency extractors --disable-dependency-tracking speeds up one-time build --enable-shared[=PKGS] build shared libraries [default=yes] --enable-static[=PKGS] build static libraries [default=yes] --enable-fast-install[=PKGS] optimize for fast installation [default=yes] --disable-libtool-lock avoid locking (might break parallel builds) --enable-nitpicking make compiler warn about possible problems --enable-parallelization enable openmp --enable-profiling add -pg to C++ compiler flags --enable-debug Enable debugging, disable optimization in the compiler --enable-libc++ use clang's libc++ vs. g++ libstdc++ headers/libraries Optional Packages: --with-PACKAGE[=ARG] use PACKAGE [ARG=yes] --without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no) --with-pic[=PKGS] try to use only PIC/non-PIC objects [default=use both] --with-aix-soname=aix|svr4|both shared library versioning (aka "SONAME") variant to provide on AIX, [default=aix]. --with-gnu-ld assume the C compiler uses GNU ld [default=no] --with-sysroot[=DIR] Search for dependent libraries within DIR (or the compiler's sysroot if not specified). --with-boost[=ARG] use Boost library from a standard location (ARG=yes), from the specified location (ARG=), or disable it (ARG=no) [ARG=yes] --with-boost-libdir=LIB_DIR Force given directory for boost libraries. Note that this will override library path detection, so use this parameter only if default library detection fails and you know exactly where your boost libraries are located. --with-boost-program-options[=special-lib] use the program options library from boost - it is possible to specify a certain library for the linker e.g. --with-boost-program-options=boost_program_options-gcc-mt-1_33_1 --with-zlib=DIR root directory path of zlib installation defaults to /usr/local or /usr if not found in /usr/local --without-zlib to disable zlib usage completely Some influential environment variables: CXX C++ compiler command CXXFLAGS C++ compiler flags LDFLAGS linker flags, e.g. -L if you have libraries in a nonstandard directory LIBS libraries to pass to the linker, e.g. -l CPPFLAGS (Objective) C/C++ preprocessor flags, e.g. -I if you have headers in a nonstandard directory CXXCPP C++ preprocessor CC C compiler command CFLAGS C compiler flags LT_SYS_LIBRARY_PATH User-defined run-time library search path. CPP C preprocessor Use these variables to override the choices made by `configure' or to help it to find libraries and programs with nonstandard names/locations. Report bugs to . vowpal_wabbit home page: . _ACEOF ac_status=$? fi if test "$ac_init_help" = "recursive"; then # If there are subdirs, report their specific --help. for ac_dir in : $ac_subdirs_all; do test "x$ac_dir" = x: && continue test -d "$ac_dir" || { cd "$srcdir" && ac_pwd=`pwd` && srcdir=. && test -d "$ac_dir"; } || continue ac_builddir=. case "$ac_dir" in .) ac_dir_suffix= ac_top_builddir_sub=. ac_top_build_prefix= ;; *) ac_dir_suffix=/`$as_echo "$ac_dir" | sed 's|^\.[\\/]||'` # A ".." for each directory in $ac_dir_suffix. ac_top_builddir_sub=`$as_echo "$ac_dir_suffix" | sed 's|/[^\\/]*|/..|g;s|/||'` case $ac_top_builddir_sub in "") ac_top_builddir_sub=. ac_top_build_prefix= ;; *) ac_top_build_prefix=$ac_top_builddir_sub/ ;; esac ;; esac ac_abs_top_builddir=$ac_pwd ac_abs_builddir=$ac_pwd$ac_dir_suffix # for backward compatibility: ac_top_builddir=$ac_top_build_prefix case $srcdir in .) # We are building in place. ac_srcdir=. ac_top_srcdir=$ac_top_builddir_sub ac_abs_top_srcdir=$ac_pwd ;; [\\/]* | ?:[\\/]* ) # Absolute name. ac_srcdir=$srcdir$ac_dir_suffix; ac_top_srcdir=$srcdir ac_abs_top_srcdir=$srcdir ;; *) # Relative name. ac_srcdir=$ac_top_build_prefix$srcdir$ac_dir_suffix ac_top_srcdir=$ac_top_build_prefix$srcdir ac_abs_top_srcdir=$ac_pwd/$srcdir ;; esac ac_abs_srcdir=$ac_abs_top_srcdir$ac_dir_suffix cd "$ac_dir" || { ac_status=$?; continue; } # Check for guested configure. if test -f "$ac_srcdir/configure.gnu"; then echo && $SHELL "$ac_srcdir/configure.gnu" --help=recursive elif test -f "$ac_srcdir/configure"; then echo && $SHELL "$ac_srcdir/configure" --help=recursive else $as_echo "$as_me: WARNING: no configuration information is in $ac_dir" >&2 fi || ac_status=$? cd "$ac_pwd" || { ac_status=$?; break; } done fi test -n "$ac_init_help" && exit $ac_status if $ac_init_version; then cat <<\_ACEOF vowpal_wabbit configure 8.5.0 generated by GNU Autoconf 2.69 Copyright (C) 2012 Free Software Foundation, Inc. This configure script is free software; the Free Software Foundation gives unlimited permission to copy, distribute and modify it. _ACEOF exit fi ## ------------------------ ## ## Autoconf initialization. ## ## ------------------------ ## # ac_fn_cxx_try_compile LINENO # ---------------------------- # Try to compile conftest.$ac_ext, and return whether this succeeded. ac_fn_cxx_try_compile () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack rm -f conftest.$ac_objext if { { ac_try="$ac_compile" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_compile") 2>conftest.err ac_status=$? if test -s conftest.err; then grep -v '^ *+' conftest.err >conftest.er1 cat conftest.er1 >&5 mv -f conftest.er1 conftest.err fi $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } && { test -z "$ac_cxx_werror_flag" || test ! -s conftest.err } && test -s conftest.$ac_objext; then : ac_retval=0 else $as_echo "$as_me: failed program was:" >&5 sed 's/^/| /' conftest.$ac_ext >&5 ac_retval=1 fi eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno as_fn_set_status $ac_retval } # ac_fn_cxx_try_compile # ac_fn_cxx_try_cpp LINENO # ------------------------ # Try to preprocess conftest.$ac_ext, and return whether this succeeded. ac_fn_cxx_try_cpp () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack if { { ac_try="$ac_cpp conftest.$ac_ext" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_cpp conftest.$ac_ext") 2>conftest.err ac_status=$? if test -s conftest.err; then grep -v '^ *+' conftest.err >conftest.er1 cat conftest.er1 >&5 mv -f conftest.er1 conftest.err fi $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } > conftest.i && { test -z "$ac_cxx_preproc_warn_flag$ac_cxx_werror_flag" || test ! -s conftest.err }; then : ac_retval=0 else $as_echo "$as_me: failed program was:" >&5 sed 's/^/| /' conftest.$ac_ext >&5 ac_retval=1 fi eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno as_fn_set_status $ac_retval } # ac_fn_cxx_try_cpp # ac_fn_c_try_compile LINENO # -------------------------- # Try to compile conftest.$ac_ext, and return whether this succeeded. ac_fn_c_try_compile () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack rm -f conftest.$ac_objext if { { ac_try="$ac_compile" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_compile") 2>conftest.err ac_status=$? if test -s conftest.err; then grep -v '^ *+' conftest.err >conftest.er1 cat conftest.er1 >&5 mv -f conftest.er1 conftest.err fi $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } && { test -z "$ac_c_werror_flag" || test ! -s conftest.err } && test -s conftest.$ac_objext; then : ac_retval=0 else $as_echo "$as_me: failed program was:" >&5 sed 's/^/| /' conftest.$ac_ext >&5 ac_retval=1 fi eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno as_fn_set_status $ac_retval } # ac_fn_c_try_compile # ac_fn_c_try_link LINENO # ----------------------- # Try to link conftest.$ac_ext, and return whether this succeeded. ac_fn_c_try_link () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack rm -f conftest.$ac_objext conftest$ac_exeext if { { ac_try="$ac_link" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_link") 2>conftest.err ac_status=$? if test -s conftest.err; then grep -v '^ *+' conftest.err >conftest.er1 cat conftest.er1 >&5 mv -f conftest.er1 conftest.err fi $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } && { test -z "$ac_c_werror_flag" || test ! -s conftest.err } && test -s conftest$ac_exeext && { test "$cross_compiling" = yes || test -x conftest$ac_exeext }; then : ac_retval=0 else $as_echo "$as_me: failed program was:" >&5 sed 's/^/| /' conftest.$ac_ext >&5 ac_retval=1 fi # Delete the IPA/IPO (Inter Procedural Analysis/Optimization) information # created by the PGI compiler (conftest_ipa8_conftest.oo), as it would # interfere with the next link command; also delete a directory that is # left behind by Apple's compiler. We do this before executing the actions. rm -rf conftest.dSYM conftest_ipa8_conftest.oo eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno as_fn_set_status $ac_retval } # ac_fn_c_try_link # ac_fn_c_check_header_compile LINENO HEADER VAR INCLUDES # ------------------------------------------------------- # Tests whether HEADER exists and can be compiled using the include files in # INCLUDES, setting the cache variable VAR accordingly. ac_fn_c_check_header_compile () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $2" >&5 $as_echo_n "checking for $2... " >&6; } if eval \${$3+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ $4 #include <$2> _ACEOF if ac_fn_c_try_compile "$LINENO"; then : eval "$3=yes" else eval "$3=no" fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi eval ac_res=\$$3 { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 $as_echo "$ac_res" >&6; } eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno } # ac_fn_c_check_header_compile # ac_fn_c_try_cpp LINENO # ---------------------- # Try to preprocess conftest.$ac_ext, and return whether this succeeded. ac_fn_c_try_cpp () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack if { { ac_try="$ac_cpp conftest.$ac_ext" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_cpp conftest.$ac_ext") 2>conftest.err ac_status=$? if test -s conftest.err; then grep -v '^ *+' conftest.err >conftest.er1 cat conftest.er1 >&5 mv -f conftest.er1 conftest.err fi $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } > conftest.i && { test -z "$ac_c_preproc_warn_flag$ac_c_werror_flag" || test ! -s conftest.err }; then : ac_retval=0 else $as_echo "$as_me: failed program was:" >&5 sed 's/^/| /' conftest.$ac_ext >&5 ac_retval=1 fi eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno as_fn_set_status $ac_retval } # ac_fn_c_try_cpp # ac_fn_c_try_run LINENO # ---------------------- # Try to link conftest.$ac_ext, and return whether this succeeded. Assumes # that executables *can* be run. ac_fn_c_try_run () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack if { { ac_try="$ac_link" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_link") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } && { ac_try='./conftest$ac_exeext' { { case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_try") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; }; then : ac_retval=0 else $as_echo "$as_me: program exited with status $ac_status" >&5 $as_echo "$as_me: failed program was:" >&5 sed 's/^/| /' conftest.$ac_ext >&5 ac_retval=$ac_status fi rm -rf conftest.dSYM conftest_ipa8_conftest.oo eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno as_fn_set_status $ac_retval } # ac_fn_c_try_run # ac_fn_c_check_func LINENO FUNC VAR # ---------------------------------- # Tests whether FUNC exists, setting the cache variable VAR accordingly ac_fn_c_check_func () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $2" >&5 $as_echo_n "checking for $2... " >&6; } if eval \${$3+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ /* Define $2 to an innocuous variant, in case declares $2. For example, HP-UX 11i declares gettimeofday. */ #define $2 innocuous_$2 /* System header to define __stub macros and hopefully few prototypes, which can conflict with char $2 (); below. Prefer to if __STDC__ is defined, since exists even on freestanding compilers. */ #ifdef __STDC__ # include #else # include #endif #undef $2 /* Override any GCC internal prototype to avoid an error. Use char because int might match the return type of a GCC builtin and then its argument prototype would still apply. */ #ifdef __cplusplus extern "C" #endif char $2 (); /* The GNU C library defines this for functions which it implements to always fail with ENOSYS. Some functions are actually named something starting with __ and the normal name is an alias. */ #if defined __stub_$2 || defined __stub___$2 choke me #endif int main () { return $2 (); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : eval "$3=yes" else eval "$3=no" fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi eval ac_res=\$$3 { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 $as_echo "$ac_res" >&6; } eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno } # ac_fn_c_check_func # ac_fn_cxx_try_link LINENO # ------------------------- # Try to link conftest.$ac_ext, and return whether this succeeded. ac_fn_cxx_try_link () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack rm -f conftest.$ac_objext conftest$ac_exeext if { { ac_try="$ac_link" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_link") 2>conftest.err ac_status=$? if test -s conftest.err; then grep -v '^ *+' conftest.err >conftest.er1 cat conftest.er1 >&5 mv -f conftest.er1 conftest.err fi $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } && { test -z "$ac_cxx_werror_flag" || test ! -s conftest.err } && test -s conftest$ac_exeext && { test "$cross_compiling" = yes || test -x conftest$ac_exeext }; then : ac_retval=0 else $as_echo "$as_me: failed program was:" >&5 sed 's/^/| /' conftest.$ac_ext >&5 ac_retval=1 fi # Delete the IPA/IPO (Inter Procedural Analysis/Optimization) information # created by the PGI compiler (conftest_ipa8_conftest.oo), as it would # interfere with the next link command; also delete a directory that is # left behind by Apple's compiler. We do this before executing the actions. rm -rf conftest.dSYM conftest_ipa8_conftest.oo eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno as_fn_set_status $ac_retval } # ac_fn_cxx_try_link # ac_fn_cxx_try_run LINENO # ------------------------ # Try to link conftest.$ac_ext, and return whether this succeeded. Assumes # that executables *can* be run. ac_fn_cxx_try_run () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack if { { ac_try="$ac_link" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_link") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } && { ac_try='./conftest$ac_exeext' { { case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_try") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; }; then : ac_retval=0 else $as_echo "$as_me: program exited with status $ac_status" >&5 $as_echo "$as_me: failed program was:" >&5 sed 's/^/| /' conftest.$ac_ext >&5 ac_retval=$ac_status fi rm -rf conftest.dSYM conftest_ipa8_conftest.oo eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno as_fn_set_status $ac_retval } # ac_fn_cxx_try_run # ac_fn_c_check_header_mongrel LINENO HEADER VAR INCLUDES # ------------------------------------------------------- # Tests whether HEADER exists, giving a warning if it cannot be compiled using # the include files in INCLUDES and setting the cache variable VAR # accordingly. ac_fn_c_check_header_mongrel () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack if eval \${$3+:} false; then : { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $2" >&5 $as_echo_n "checking for $2... " >&6; } if eval \${$3+:} false; then : $as_echo_n "(cached) " >&6 fi eval ac_res=\$$3 { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 $as_echo "$ac_res" >&6; } else # Is the header compilable? { $as_echo "$as_me:${as_lineno-$LINENO}: checking $2 usability" >&5 $as_echo_n "checking $2 usability... " >&6; } cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ $4 #include <$2> _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_header_compiler=yes else ac_header_compiler=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_header_compiler" >&5 $as_echo "$ac_header_compiler" >&6; } # Is the header present? { $as_echo "$as_me:${as_lineno-$LINENO}: checking $2 presence" >&5 $as_echo_n "checking $2 presence... " >&6; } cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include <$2> _ACEOF if ac_fn_c_try_cpp "$LINENO"; then : ac_header_preproc=yes else ac_header_preproc=no fi rm -f conftest.err conftest.i conftest.$ac_ext { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_header_preproc" >&5 $as_echo "$ac_header_preproc" >&6; } # So? What about this header? case $ac_header_compiler:$ac_header_preproc:$ac_c_preproc_warn_flag in #(( yes:no: ) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: accepted by the compiler, rejected by the preprocessor!" >&5 $as_echo "$as_me: WARNING: $2: accepted by the compiler, rejected by the preprocessor!" >&2;} { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: proceeding with the compiler's result" >&5 $as_echo "$as_me: WARNING: $2: proceeding with the compiler's result" >&2;} ;; no:yes:* ) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: present but cannot be compiled" >&5 $as_echo "$as_me: WARNING: $2: present but cannot be compiled" >&2;} { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: check for missing prerequisite headers?" >&5 $as_echo "$as_me: WARNING: $2: check for missing prerequisite headers?" >&2;} { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: see the Autoconf documentation" >&5 $as_echo "$as_me: WARNING: $2: see the Autoconf documentation" >&2;} { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: section \"Present But Cannot Be Compiled\"" >&5 $as_echo "$as_me: WARNING: $2: section \"Present But Cannot Be Compiled\"" >&2;} { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: proceeding with the compiler's result" >&5 $as_echo "$as_me: WARNING: $2: proceeding with the compiler's result" >&2;} ( $as_echo "## --------------------------- ## ## Report this to jl@hunch.net ## ## --------------------------- ##" ) | sed "s/^/$as_me: WARNING: /" >&2 ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $2" >&5 $as_echo_n "checking for $2... " >&6; } if eval \${$3+:} false; then : $as_echo_n "(cached) " >&6 else eval "$3=\$ac_header_compiler" fi eval ac_res=\$$3 { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 $as_echo "$ac_res" >&6; } fi eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno } # ac_fn_c_check_header_mongrel cat >config.log <<_ACEOF This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. It was created by vowpal_wabbit $as_me 8.5.0, which was generated by GNU Autoconf 2.69. Invocation command line was $ $0 $@ _ACEOF exec 5>>config.log { cat <<_ASUNAME ## --------- ## ## Platform. ## ## --------- ## hostname = `(hostname || uname -n) 2>/dev/null | sed 1q` uname -m = `(uname -m) 2>/dev/null || echo unknown` uname -r = `(uname -r) 2>/dev/null || echo unknown` uname -s = `(uname -s) 2>/dev/null || echo unknown` uname -v = `(uname -v) 2>/dev/null || echo unknown` /usr/bin/uname -p = `(/usr/bin/uname -p) 2>/dev/null || echo unknown` /bin/uname -X = `(/bin/uname -X) 2>/dev/null || echo unknown` /bin/arch = `(/bin/arch) 2>/dev/null || echo unknown` /usr/bin/arch -k = `(/usr/bin/arch -k) 2>/dev/null || echo unknown` /usr/convex/getsysinfo = `(/usr/convex/getsysinfo) 2>/dev/null || echo unknown` /usr/bin/hostinfo = `(/usr/bin/hostinfo) 2>/dev/null || echo unknown` /bin/machine = `(/bin/machine) 2>/dev/null || echo unknown` /usr/bin/oslevel = `(/usr/bin/oslevel) 2>/dev/null || echo unknown` /bin/universe = `(/bin/universe) 2>/dev/null || echo unknown` _ASUNAME as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. $as_echo "PATH: $as_dir" done IFS=$as_save_IFS } >&5 cat >&5 <<_ACEOF ## ----------- ## ## Core tests. ## ## ----------- ## _ACEOF # Keep a trace of the command line. # Strip out --no-create and --no-recursion so they do not pile up. # Strip out --silent because we don't want to record it for future runs. # Also quote any args containing shell meta-characters. # Make two passes to allow for proper duplicate-argument suppression. ac_configure_args= ac_configure_args0= ac_configure_args1= ac_must_keep_next=false for ac_pass in 1 2 do for ac_arg do case $ac_arg in -no-create | --no-c* | -n | -no-recursion | --no-r*) continue ;; -q | -quiet | --quiet | --quie | --qui | --qu | --q \ | -silent | --silent | --silen | --sile | --sil) continue ;; *\'*) ac_arg=`$as_echo "$ac_arg" | sed "s/'/'\\\\\\\\''/g"` ;; esac case $ac_pass in 1) as_fn_append ac_configure_args0 " '$ac_arg'" ;; 2) as_fn_append ac_configure_args1 " '$ac_arg'" if test $ac_must_keep_next = true; then ac_must_keep_next=false # Got value, back to normal. else case $ac_arg in *=* | --config-cache | -C | -disable-* | --disable-* \ | -enable-* | --enable-* | -gas | --g* | -nfp | --nf* \ | -q | -quiet | --q* | -silent | --sil* | -v | -verb* \ | -with-* | --with-* | -without-* | --without-* | --x) case "$ac_configure_args0 " in "$ac_configure_args1"*" '$ac_arg' "* ) continue ;; esac ;; -* ) ac_must_keep_next=true ;; esac fi as_fn_append ac_configure_args " '$ac_arg'" ;; esac done done { ac_configure_args0=; unset ac_configure_args0;} { ac_configure_args1=; unset ac_configure_args1;} # When interrupted or exit'd, cleanup temporary files, and complete # config.log. We remove comments because anyway the quotes in there # would cause problems or look ugly. # WARNING: Use '\'' to represent an apostrophe within the trap. # WARNING: Do not start the trap code with a newline, due to a FreeBSD 4.0 bug. trap 'exit_status=$? # Save into config.log some information that might help in debugging. { echo $as_echo "## ---------------- ## ## Cache variables. ## ## ---------------- ##" echo # The following way of writing the cache mishandles newlines in values, ( for ac_var in `(set) 2>&1 | sed -n '\''s/^\([a-zA-Z_][a-zA-Z0-9_]*\)=.*/\1/p'\''`; do eval ac_val=\$$ac_var case $ac_val in #( *${as_nl}*) case $ac_var in #( *_cv_*) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: cache variable $ac_var contains a newline" >&5 $as_echo "$as_me: WARNING: cache variable $ac_var contains a newline" >&2;} ;; esac case $ac_var in #( _ | IFS | as_nl) ;; #( BASH_ARGV | BASH_SOURCE) eval $ac_var= ;; #( *) { eval $ac_var=; unset $ac_var;} ;; esac ;; esac done (set) 2>&1 | case $as_nl`(ac_space='\'' '\''; set) 2>&1` in #( *${as_nl}ac_space=\ *) sed -n \ "s/'\''/'\''\\\\'\'''\''/g; s/^\\([_$as_cr_alnum]*_cv_[_$as_cr_alnum]*\\)=\\(.*\\)/\\1='\''\\2'\''/p" ;; #( *) sed -n "/^[_$as_cr_alnum]*_cv_[_$as_cr_alnum]*=/p" ;; esac | sort ) echo $as_echo "## ----------------- ## ## Output variables. ## ## ----------------- ##" echo for ac_var in $ac_subst_vars do eval ac_val=\$$ac_var case $ac_val in *\'\''*) ac_val=`$as_echo "$ac_val" | sed "s/'\''/'\''\\\\\\\\'\'''\''/g"`;; esac $as_echo "$ac_var='\''$ac_val'\''" done | sort echo if test -n "$ac_subst_files"; then $as_echo "## ------------------- ## ## File substitutions. ## ## ------------------- ##" echo for ac_var in $ac_subst_files do eval ac_val=\$$ac_var case $ac_val in *\'\''*) ac_val=`$as_echo "$ac_val" | sed "s/'\''/'\''\\\\\\\\'\'''\''/g"`;; esac $as_echo "$ac_var='\''$ac_val'\''" done | sort echo fi if test -s confdefs.h; then $as_echo "## ----------- ## ## confdefs.h. ## ## ----------- ##" echo cat confdefs.h echo fi test "$ac_signal" != 0 && $as_echo "$as_me: caught signal $ac_signal" $as_echo "$as_me: exit $exit_status" } >&5 rm -f core *.core core.conftest.* && rm -f -r conftest* confdefs* conf$$* $ac_clean_files && exit $exit_status ' 0 for ac_signal in 1 2 13 15; do trap 'ac_signal='$ac_signal'; as_fn_exit 1' $ac_signal done ac_signal=0 # confdefs.h avoids OS command line length limits that DEFS can exceed. rm -f -r conftest* confdefs.h $as_echo "/* confdefs.h */" > confdefs.h # Predefined preprocessor variables. cat >>confdefs.h <<_ACEOF #define PACKAGE_NAME "$PACKAGE_NAME" _ACEOF cat >>confdefs.h <<_ACEOF #define PACKAGE_TARNAME "$PACKAGE_TARNAME" _ACEOF cat >>confdefs.h <<_ACEOF #define PACKAGE_VERSION "$PACKAGE_VERSION" _ACEOF cat >>confdefs.h <<_ACEOF #define PACKAGE_STRING "$PACKAGE_STRING" _ACEOF cat >>confdefs.h <<_ACEOF #define PACKAGE_BUGREPORT "$PACKAGE_BUGREPORT" _ACEOF cat >>confdefs.h <<_ACEOF #define PACKAGE_URL "$PACKAGE_URL" _ACEOF # Let the site file select an alternate cache file if it wants to. # Prefer an explicitly selected file to automatically selected ones. ac_site_file1=NONE ac_site_file2=NONE if test -n "$CONFIG_SITE"; then # We do not want a PATH search for config.site. case $CONFIG_SITE in #(( -*) ac_site_file1=./$CONFIG_SITE;; */*) ac_site_file1=$CONFIG_SITE;; *) ac_site_file1=./$CONFIG_SITE;; esac elif test "x$prefix" != xNONE; then ac_site_file1=$prefix/share/config.site ac_site_file2=$prefix/etc/config.site else ac_site_file1=$ac_default_prefix/share/config.site ac_site_file2=$ac_default_prefix/etc/config.site fi for ac_site_file in "$ac_site_file1" "$ac_site_file2" do test "x$ac_site_file" = xNONE && continue if test /dev/null != "$ac_site_file" && test -r "$ac_site_file"; then { $as_echo "$as_me:${as_lineno-$LINENO}: loading site script $ac_site_file" >&5 $as_echo "$as_me: loading site script $ac_site_file" >&6;} sed 's/^/| /' "$ac_site_file" >&5 . "$ac_site_file" \ || { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 $as_echo "$as_me: error: in \`$ac_pwd':" >&2;} as_fn_error $? "failed to load site script $ac_site_file See \`config.log' for more details" "$LINENO" 5; } fi done if test -r "$cache_file"; then # Some versions of bash will fail to source /dev/null (special files # actually), so we avoid doing that. DJGPP emulates it as a regular file. if test /dev/null != "$cache_file" && test -f "$cache_file"; then { $as_echo "$as_me:${as_lineno-$LINENO}: loading cache $cache_file" >&5 $as_echo "$as_me: loading cache $cache_file" >&6;} case $cache_file in [\\/]* | ?:[\\/]* ) . "$cache_file";; *) . "./$cache_file";; esac fi else { $as_echo "$as_me:${as_lineno-$LINENO}: creating cache $cache_file" >&5 $as_echo "$as_me: creating cache $cache_file" >&6;} >$cache_file fi # Check that the precious variables saved in the cache have kept the same # value. ac_cache_corrupted=false for ac_var in $ac_precious_vars; do eval ac_old_set=\$ac_cv_env_${ac_var}_set eval ac_new_set=\$ac_env_${ac_var}_set eval ac_old_val=\$ac_cv_env_${ac_var}_value eval ac_new_val=\$ac_env_${ac_var}_value case $ac_old_set,$ac_new_set in set,) { $as_echo "$as_me:${as_lineno-$LINENO}: error: \`$ac_var' was set to \`$ac_old_val' in the previous run" >&5 $as_echo "$as_me: error: \`$ac_var' was set to \`$ac_old_val' in the previous run" >&2;} ac_cache_corrupted=: ;; ,set) { $as_echo "$as_me:${as_lineno-$LINENO}: error: \`$ac_var' was not set in the previous run" >&5 $as_echo "$as_me: error: \`$ac_var' was not set in the previous run" >&2;} ac_cache_corrupted=: ;; ,);; *) if test "x$ac_old_val" != "x$ac_new_val"; then # differences in whitespace do not lead to failure. ac_old_val_w=`echo x $ac_old_val` ac_new_val_w=`echo x $ac_new_val` if test "$ac_old_val_w" != "$ac_new_val_w"; then { $as_echo "$as_me:${as_lineno-$LINENO}: error: \`$ac_var' has changed since the previous run:" >&5 $as_echo "$as_me: error: \`$ac_var' has changed since the previous run:" >&2;} ac_cache_corrupted=: else { $as_echo "$as_me:${as_lineno-$LINENO}: warning: ignoring whitespace changes in \`$ac_var' since the previous run:" >&5 $as_echo "$as_me: warning: ignoring whitespace changes in \`$ac_var' since the previous run:" >&2;} eval $ac_var=\$ac_old_val fi { $as_echo "$as_me:${as_lineno-$LINENO}: former value: \`$ac_old_val'" >&5 $as_echo "$as_me: former value: \`$ac_old_val'" >&2;} { $as_echo "$as_me:${as_lineno-$LINENO}: current value: \`$ac_new_val'" >&5 $as_echo "$as_me: current value: \`$ac_new_val'" >&2;} fi;; esac # Pass precious variables to config.status. if test "$ac_new_set" = set; then case $ac_new_val in *\'*) ac_arg=$ac_var=`$as_echo "$ac_new_val" | sed "s/'/'\\\\\\\\''/g"` ;; *) ac_arg=$ac_var=$ac_new_val ;; esac case " $ac_configure_args " in *" '$ac_arg' "*) ;; # Avoid dups. Use of quotes ensures accuracy. *) as_fn_append ac_configure_args " '$ac_arg'" ;; esac fi done if $ac_cache_corrupted; then { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 $as_echo "$as_me: error: in \`$ac_pwd':" >&2;} { $as_echo "$as_me:${as_lineno-$LINENO}: error: changes in the environment can compromise the build" >&5 $as_echo "$as_me: error: changes in the environment can compromise the build" >&2;} as_fn_error $? "run \`make distclean' and/or \`rm $cache_file' and start over" "$LINENO" 5 fi ## -------------------- ## ## Main body of script. ## ## -------------------- ## ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu ac_config_headers="$ac_config_headers vowpalwabbit/config.h" am__api_version='1.15' ac_aux_dir= for ac_dir in "$srcdir" "$srcdir/.." "$srcdir/../.."; do if test -f "$ac_dir/install-sh"; then ac_aux_dir=$ac_dir ac_install_sh="$ac_aux_dir/install-sh -c" break elif test -f "$ac_dir/install.sh"; then ac_aux_dir=$ac_dir ac_install_sh="$ac_aux_dir/install.sh -c" break elif test -f "$ac_dir/shtool"; then ac_aux_dir=$ac_dir ac_install_sh="$ac_aux_dir/shtool install -c" break fi done if test -z "$ac_aux_dir"; then as_fn_error $? "cannot find install-sh, install.sh, or shtool in \"$srcdir\" \"$srcdir/..\" \"$srcdir/../..\"" "$LINENO" 5 fi # These three variables are undocumented and unsupported, # and are intended to be withdrawn in a future Autoconf release. # They can cause serious problems if a builder's source tree is in a directory # whose full name contains unusual characters. ac_config_guess="$SHELL $ac_aux_dir/config.guess" # Please don't use this var. ac_config_sub="$SHELL $ac_aux_dir/config.sub" # Please don't use this var. ac_configure="$SHELL $ac_aux_dir/configure" # Please don't use this var. # Find a good install program. We prefer a C program (faster), # so one script is as good as another. But avoid the broken or # incompatible versions: # SysV /etc/install, /usr/sbin/install # SunOS /usr/etc/install # IRIX /sbin/install # AIX /bin/install # AmigaOS /C/install, which installs bootblocks on floppy discs # AIX 4 /usr/bin/installbsd, which doesn't work without a -g flag # AFS /usr/afsws/bin/install, which mishandles nonexistent args # SVR4 /usr/ucb/install, which tries to use the nonexistent group "staff" # OS/2's system install, which has a completely different semantic # ./install, which can be erroneously created by make from ./install.sh. # Reject install programs that cannot install multiple files. { $as_echo "$as_me:${as_lineno-$LINENO}: checking for a BSD-compatible install" >&5 $as_echo_n "checking for a BSD-compatible install... " >&6; } if test -z "$INSTALL"; then if ${ac_cv_path_install+:} false; then : $as_echo_n "(cached) " >&6 else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. # Account for people who put trailing slashes in PATH elements. case $as_dir/ in #(( ./ | .// | /[cC]/* | \ /etc/* | /usr/sbin/* | /usr/etc/* | /sbin/* | /usr/afsws/bin/* | \ ?:[\\/]os2[\\/]install[\\/]* | ?:[\\/]OS2[\\/]INSTALL[\\/]* | \ /usr/ucb/* ) ;; *) # OSF1 and SCO ODT 3.0 have their own names for install. # Don't use installbsd from OSF since it installs stuff as root # by default. for ac_prog in ginstall scoinst install; do for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_prog$ac_exec_ext"; then if test $ac_prog = install && grep dspmsg "$as_dir/$ac_prog$ac_exec_ext" >/dev/null 2>&1; then # AIX install. It has an incompatible calling convention. : elif test $ac_prog = install && grep pwplus "$as_dir/$ac_prog$ac_exec_ext" >/dev/null 2>&1; then # program-specific install script used by HP pwplus--don't use. : else rm -rf conftest.one conftest.two conftest.dir echo one > conftest.one echo two > conftest.two mkdir conftest.dir if "$as_dir/$ac_prog$ac_exec_ext" -c conftest.one conftest.two "`pwd`/conftest.dir" && test -s conftest.one && test -s conftest.two && test -s conftest.dir/conftest.one && test -s conftest.dir/conftest.two then ac_cv_path_install="$as_dir/$ac_prog$ac_exec_ext -c" break 3 fi fi fi done done ;; esac done IFS=$as_save_IFS rm -rf conftest.one conftest.two conftest.dir fi if test "${ac_cv_path_install+set}" = set; then INSTALL=$ac_cv_path_install else # As a last resort, use the slow shell script. Don't cache a # value for INSTALL within a source directory, because that will # break other packages using the cache if that directory is # removed, or if the value is a relative name. INSTALL=$ac_install_sh fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $INSTALL" >&5 $as_echo "$INSTALL" >&6; } # Use test -z because SunOS4 sh mishandles braces in ${var-val}. # It thinks the first close brace ends the variable substitution. test -z "$INSTALL_PROGRAM" && INSTALL_PROGRAM='${INSTALL}' test -z "$INSTALL_SCRIPT" && INSTALL_SCRIPT='${INSTALL}' test -z "$INSTALL_DATA" && INSTALL_DATA='${INSTALL} -m 644' { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether build environment is sane" >&5 $as_echo_n "checking whether build environment is sane... " >&6; } # Reject unsafe characters in $srcdir or the absolute working directory # name. Accept space and tab only in the latter. am_lf=' ' case `pwd` in *[\\\"\#\$\&\'\`$am_lf]*) as_fn_error $? "unsafe absolute working directory name" "$LINENO" 5;; esac case $srcdir in *[\\\"\#\$\&\'\`$am_lf\ \ ]*) as_fn_error $? "unsafe srcdir value: '$srcdir'" "$LINENO" 5;; esac # Do 'set' in a subshell so we don't clobber the current shell's # arguments. Must try -L first in case configure is actually a # symlink; some systems play weird games with the mod time of symlinks # (eg FreeBSD returns the mod time of the symlink's containing # directory). if ( am_has_slept=no for am_try in 1 2; do echo "timestamp, slept: $am_has_slept" > conftest.file set X `ls -Lt "$srcdir/configure" conftest.file 2> /dev/null` if test "$*" = "X"; then # -L didn't work. set X `ls -t "$srcdir/configure" conftest.file` fi if test "$*" != "X $srcdir/configure conftest.file" \ && test "$*" != "X conftest.file $srcdir/configure"; then # If neither matched, then we have a broken ls. This can happen # if, for instance, CONFIG_SHELL is bash and it inherits a # broken ls alias from the environment. This has actually # happened. Such a system could not be considered "sane". as_fn_error $? "ls -t appears to fail. Make sure there is not a broken alias in your environment" "$LINENO" 5 fi if test "$2" = conftest.file || test $am_try -eq 2; then break fi # Just in case. sleep 1 am_has_slept=yes done test "$2" = conftest.file ) then # Ok. : else as_fn_error $? "newly created file is older than distributed files! Check your system clock" "$LINENO" 5 fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } # If we didn't sleep, we still need to ensure time stamps of config.status and # generated files are strictly newer. am_sleep_pid= if grep 'slept: no' conftest.file >/dev/null 2>&1; then ( sleep 1 ) & am_sleep_pid=$! fi rm -f conftest.file test "$program_prefix" != NONE && program_transform_name="s&^&$program_prefix&;$program_transform_name" # Use a double $ so make ignores it. test "$program_suffix" != NONE && program_transform_name="s&\$&$program_suffix&;$program_transform_name" # Double any \ or $. # By default was `s,x,x', remove it if useless. ac_script='s/[\\$]/&&/g;s/;s,x,x,$//' program_transform_name=`$as_echo "$program_transform_name" | sed "$ac_script"` # Expand $ac_aux_dir to an absolute path. am_aux_dir=`cd "$ac_aux_dir" && pwd` if test x"${MISSING+set}" != xset; then case $am_aux_dir in *\ * | *\ *) MISSING="\${SHELL} \"$am_aux_dir/missing\"" ;; *) MISSING="\${SHELL} $am_aux_dir/missing" ;; esac fi # Use eval to expand $SHELL if eval "$MISSING --is-lightweight"; then am_missing_run="$MISSING " else am_missing_run= { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: 'missing' script is too old or missing" >&5 $as_echo "$as_me: WARNING: 'missing' script is too old or missing" >&2;} fi if test x"${install_sh+set}" != xset; then case $am_aux_dir in *\ * | *\ *) install_sh="\${SHELL} '$am_aux_dir/install-sh'" ;; *) install_sh="\${SHELL} $am_aux_dir/install-sh" esac fi # Installed binaries are usually stripped using 'strip' when the user # run "make install-strip". However 'strip' might not be the right # tool to use in cross-compilation environments, therefore Automake # will honor the 'STRIP' environment variable to overrule this program. if test "$cross_compiling" != no; then if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}strip", so it can be a program name with args. set dummy ${ac_tool_prefix}strip; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_STRIP+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$STRIP"; then ac_cv_prog_STRIP="$STRIP" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_STRIP="${ac_tool_prefix}strip" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi STRIP=$ac_cv_prog_STRIP if test -n "$STRIP"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $STRIP" >&5 $as_echo "$STRIP" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_STRIP"; then ac_ct_STRIP=$STRIP # Extract the first word of "strip", so it can be a program name with args. set dummy strip; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_STRIP+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_STRIP"; then ac_cv_prog_ac_ct_STRIP="$ac_ct_STRIP" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_STRIP="strip" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_STRIP=$ac_cv_prog_ac_ct_STRIP if test -n "$ac_ct_STRIP"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_STRIP" >&5 $as_echo "$ac_ct_STRIP" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_STRIP" = x; then STRIP=":" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac STRIP=$ac_ct_STRIP fi else STRIP="$ac_cv_prog_STRIP" fi fi INSTALL_STRIP_PROGRAM="\$(install_sh) -c -s" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for a thread-safe mkdir -p" >&5 $as_echo_n "checking for a thread-safe mkdir -p... " >&6; } if test -z "$MKDIR_P"; then if ${ac_cv_path_mkdir+:} false; then : $as_echo_n "(cached) " >&6 else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH$PATH_SEPARATOR/opt/sfw/bin do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_prog in mkdir gmkdir; do for ac_exec_ext in '' $ac_executable_extensions; do as_fn_executable_p "$as_dir/$ac_prog$ac_exec_ext" || continue case `"$as_dir/$ac_prog$ac_exec_ext" --version 2>&1` in #( 'mkdir (GNU coreutils) '* | \ 'mkdir (coreutils) '* | \ 'mkdir (fileutils) '4.1*) ac_cv_path_mkdir=$as_dir/$ac_prog$ac_exec_ext break 3;; esac done done done IFS=$as_save_IFS fi test -d ./--version && rmdir ./--version if test "${ac_cv_path_mkdir+set}" = set; then MKDIR_P="$ac_cv_path_mkdir -p" else # As a last resort, use the slow shell script. Don't cache a # value for MKDIR_P within a source directory, because that will # break other packages using the cache if that directory is # removed, or if the value is a relative name. MKDIR_P="$ac_install_sh -d" fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $MKDIR_P" >&5 $as_echo "$MKDIR_P" >&6; } for ac_prog in gawk mawk nawk awk do # Extract the first word of "$ac_prog", so it can be a program name with args. set dummy $ac_prog; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_AWK+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$AWK"; then ac_cv_prog_AWK="$AWK" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_AWK="$ac_prog" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi AWK=$ac_cv_prog_AWK if test -n "$AWK"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $AWK" >&5 $as_echo "$AWK" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi test -n "$AWK" && break done { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether ${MAKE-make} sets \$(MAKE)" >&5 $as_echo_n "checking whether ${MAKE-make} sets \$(MAKE)... " >&6; } set x ${MAKE-make} ac_make=`$as_echo "$2" | sed 's/+/p/g; s/[^a-zA-Z0-9_]/_/g'` if eval \${ac_cv_prog_make_${ac_make}_set+:} false; then : $as_echo_n "(cached) " >&6 else cat >conftest.make <<\_ACEOF SHELL = /bin/sh all: @echo '@@@%%%=$(MAKE)=@@@%%%' _ACEOF # GNU make sometimes prints "make[1]: Entering ...", which would confuse us. case `${MAKE-make} -f conftest.make 2>/dev/null` in *@@@%%%=?*=@@@%%%*) eval ac_cv_prog_make_${ac_make}_set=yes;; *) eval ac_cv_prog_make_${ac_make}_set=no;; esac rm -f conftest.make fi if eval test \$ac_cv_prog_make_${ac_make}_set = yes; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } SET_MAKE= else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } SET_MAKE="MAKE=${MAKE-make}" fi rm -rf .tst 2>/dev/null mkdir .tst 2>/dev/null if test -d .tst; then am__leading_dot=. else am__leading_dot=_ fi rmdir .tst 2>/dev/null # Check whether --enable-silent-rules was given. if test "${enable_silent_rules+set}" = set; then : enableval=$enable_silent_rules; fi case $enable_silent_rules in # ((( yes) AM_DEFAULT_VERBOSITY=0;; no) AM_DEFAULT_VERBOSITY=1;; *) AM_DEFAULT_VERBOSITY=1;; esac am_make=${MAKE-make} { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether $am_make supports nested variables" >&5 $as_echo_n "checking whether $am_make supports nested variables... " >&6; } if ${am_cv_make_support_nested_variables+:} false; then : $as_echo_n "(cached) " >&6 else if $as_echo 'TRUE=$(BAR$(V)) BAR0=false BAR1=true V=1 am__doit: @$(TRUE) .PHONY: am__doit' | $am_make -f - >/dev/null 2>&1; then am_cv_make_support_nested_variables=yes else am_cv_make_support_nested_variables=no fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $am_cv_make_support_nested_variables" >&5 $as_echo "$am_cv_make_support_nested_variables" >&6; } if test $am_cv_make_support_nested_variables = yes; then AM_V='$(V)' AM_DEFAULT_V='$(AM_DEFAULT_VERBOSITY)' else AM_V=$AM_DEFAULT_VERBOSITY AM_DEFAULT_V=$AM_DEFAULT_VERBOSITY fi AM_BACKSLASH='\' if test "`cd $srcdir && pwd`" != "`pwd`"; then # Use -I$(srcdir) only when $(srcdir) != ., so that make's output # is not polluted with repeated "-I." am__isrc=' -I$(srcdir)' # test to see if srcdir already configured if test -f $srcdir/config.status; then as_fn_error $? "source directory already configured; run \"make distclean\" there first" "$LINENO" 5 fi fi # test whether we have cygpath if test -z "$CYGPATH_W"; then if (cygpath --version) >/dev/null 2>/dev/null; then CYGPATH_W='cygpath -w' else CYGPATH_W=echo fi fi # Define the identity of the package. PACKAGE='vowpal_wabbit' VERSION='8.5.0' cat >>confdefs.h <<_ACEOF #define PACKAGE "$PACKAGE" _ACEOF cat >>confdefs.h <<_ACEOF #define VERSION "$VERSION" _ACEOF # Some tools Automake needs. ACLOCAL=${ACLOCAL-"${am_missing_run}aclocal-${am__api_version}"} AUTOCONF=${AUTOCONF-"${am_missing_run}autoconf"} AUTOMAKE=${AUTOMAKE-"${am_missing_run}automake-${am__api_version}"} AUTOHEADER=${AUTOHEADER-"${am_missing_run}autoheader"} MAKEINFO=${MAKEINFO-"${am_missing_run}makeinfo"} # For better backward compatibility. To be removed once Automake 1.9.x # dies out for good. For more background, see: # # mkdir_p='$(MKDIR_P)' # We need awk for the "check" target (and possibly the TAP driver). The # system "awk" is bad on some platforms. # Always define AMTAR for backward compatibility. Yes, it's still used # in the wild :-( We should find a proper way to deprecate it ... AMTAR='$${TAR-tar}' # We'll loop over all known methods to create a tar archive until one works. _am_tools='gnutar pax cpio none' am__tar='$${TAR-tar} chof - "$$tardir"' am__untar='$${TAR-tar} xf -' # POSIX will say in a future version that running "rm -f" with no argument # is OK; and we want to be able to make that assumption in our Makefile # recipes. So use an aggressive probe to check that the usage we want is # actually supported "in the wild" to an acceptable degree. # See automake bug#10828. # To make any issue more visible, cause the running configure to be aborted # by default if the 'rm' program in use doesn't match our expectations; the # user can still override this though. if rm -f && rm -fr && rm -rf; then : OK; else cat >&2 <<'END' Oops! Your 'rm' program seems unable to run without file operands specified on the command line, even when the '-f' option is present. This is contrary to the behaviour of most rm programs out there, and not conforming with the upcoming POSIX standard: Please tell bug-automake@gnu.org about your system, including the value of your $PATH and any error possibly output before this message. This can help us improve future automake versions. END if test x"$ACCEPT_INFERIOR_RM_PROGRAM" = x"yes"; then echo 'Configuration will proceed anyway, since you have set the' >&2 echo 'ACCEPT_INFERIOR_RM_PROGRAM variable to "yes"' >&2 echo >&2 else cat >&2 <<'END' Aborting the configuration process, to ensure you take notice of the issue. You can download and install GNU coreutils to get an 'rm' implementation that behaves properly: . If you want to complete the configuration process using your problematic 'rm' anyway, export the environment variable ACCEPT_INFERIOR_RM_PROGRAM to "yes", and re-run configure. END as_fn_error $? "Your 'rm' program is bad, sorry." "$LINENO" 5 fi fi # if CXXFLAGS is not given by user, don't use default of "-O2 -g" : ${CXXFLAGS=""} ac_ext=cpp ac_cpp='$CXXCPP $CPPFLAGS' ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_cxx_compiler_gnu if test -z "$CXX"; then if test -n "$CCC"; then CXX=$CCC else if test -n "$ac_tool_prefix"; then for ac_prog in g++ c++ gpp aCC CC cxx cc++ cl.exe FCC KCC RCC xlC_r xlC do # Extract the first word of "$ac_tool_prefix$ac_prog", so it can be a program name with args. set dummy $ac_tool_prefix$ac_prog; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_CXX+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$CXX"; then ac_cv_prog_CXX="$CXX" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_CXX="$ac_tool_prefix$ac_prog" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi CXX=$ac_cv_prog_CXX if test -n "$CXX"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $CXX" >&5 $as_echo "$CXX" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi test -n "$CXX" && break done fi if test -z "$CXX"; then ac_ct_CXX=$CXX for ac_prog in g++ c++ gpp aCC CC cxx cc++ cl.exe FCC KCC RCC xlC_r xlC do # Extract the first word of "$ac_prog", so it can be a program name with args. set dummy $ac_prog; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_CXX+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_CXX"; then ac_cv_prog_ac_ct_CXX="$ac_ct_CXX" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_CXX="$ac_prog" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_CXX=$ac_cv_prog_ac_ct_CXX if test -n "$ac_ct_CXX"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_CXX" >&5 $as_echo "$ac_ct_CXX" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi test -n "$ac_ct_CXX" && break done if test "x$ac_ct_CXX" = x; then CXX="g++" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac CXX=$ac_ct_CXX fi fi fi fi # Provide some information about the compiler. $as_echo "$as_me:${as_lineno-$LINENO}: checking for C++ compiler version" >&5 set X $ac_compile ac_compiler=$2 for ac_option in --version -v -V -qversion; do { { ac_try="$ac_compiler $ac_option >&5" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_compiler $ac_option >&5") 2>conftest.err ac_status=$? if test -s conftest.err; then sed '10a\ ... rest of stderr output deleted ... 10q' conftest.err >conftest.er1 cat conftest.er1 >&5 fi rm -f conftest.er1 conftest.err $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } done cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF ac_clean_files_save=$ac_clean_files ac_clean_files="$ac_clean_files a.out a.out.dSYM a.exe b.out" # Try to create an executable without -o first, disregard a.out. # It will help us diagnose broken compilers, and finding out an intuition # of exeext. { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether the C++ compiler works" >&5 $as_echo_n "checking whether the C++ compiler works... " >&6; } ac_link_default=`$as_echo "$ac_link" | sed 's/ -o *conftest[^ ]*//'` # The possible output files: ac_files="a.out conftest.exe conftest a.exe a_out.exe b.out conftest.*" ac_rmfiles= for ac_file in $ac_files do case $ac_file in *.$ac_ext | *.xcoff | *.tds | *.d | *.pdb | *.xSYM | *.bb | *.bbg | *.map | *.inf | *.dSYM | *.o | *.obj ) ;; * ) ac_rmfiles="$ac_rmfiles $ac_file";; esac done rm -f $ac_rmfiles if { { ac_try="$ac_link_default" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_link_default") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then : # Autoconf-2.13 could set the ac_cv_exeext variable to `no'. # So ignore a value of `no', otherwise this would lead to `EXEEXT = no' # in a Makefile. We should not override ac_cv_exeext if it was cached, # so that the user can short-circuit this test for compilers unknown to # Autoconf. for ac_file in $ac_files '' do test -f "$ac_file" || continue case $ac_file in *.$ac_ext | *.xcoff | *.tds | *.d | *.pdb | *.xSYM | *.bb | *.bbg | *.map | *.inf | *.dSYM | *.o | *.obj ) ;; [ab].out ) # We found the default executable, but exeext='' is most # certainly right. break;; *.* ) if test "${ac_cv_exeext+set}" = set && test "$ac_cv_exeext" != no; then :; else ac_cv_exeext=`expr "$ac_file" : '[^.]*\(\..*\)'` fi # We set ac_cv_exeext here because the later test for it is not # safe: cross compilers may not add the suffix if given an `-o' # argument, so we may need to know it at that point already. # Even if this section looks crufty: it has the advantage of # actually working. break;; * ) break;; esac done test "$ac_cv_exeext" = no && ac_cv_exeext= else ac_file='' fi if test -z "$ac_file"; then : { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } $as_echo "$as_me: failed program was:" >&5 sed 's/^/| /' conftest.$ac_ext >&5 { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 $as_echo "$as_me: error: in \`$ac_pwd':" >&2;} as_fn_error 77 "C++ compiler cannot create executables See \`config.log' for more details" "$LINENO" 5; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for C++ compiler default output file name" >&5 $as_echo_n "checking for C++ compiler default output file name... " >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_file" >&5 $as_echo "$ac_file" >&6; } ac_exeext=$ac_cv_exeext rm -f -r a.out a.out.dSYM a.exe conftest$ac_cv_exeext b.out ac_clean_files=$ac_clean_files_save { $as_echo "$as_me:${as_lineno-$LINENO}: checking for suffix of executables" >&5 $as_echo_n "checking for suffix of executables... " >&6; } if { { ac_try="$ac_link" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_link") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then : # If both `conftest.exe' and `conftest' are `present' (well, observable) # catch `conftest.exe'. For instance with Cygwin, `ls conftest' will # work properly (i.e., refer to `conftest.exe'), while it won't with # `rm'. for ac_file in conftest.exe conftest conftest.*; do test -f "$ac_file" || continue case $ac_file in *.$ac_ext | *.xcoff | *.tds | *.d | *.pdb | *.xSYM | *.bb | *.bbg | *.map | *.inf | *.dSYM | *.o | *.obj ) ;; *.* ) ac_cv_exeext=`expr "$ac_file" : '[^.]*\(\..*\)'` break;; * ) break;; esac done else { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 $as_echo "$as_me: error: in \`$ac_pwd':" >&2;} as_fn_error $? "cannot compute suffix of executables: cannot compile and link See \`config.log' for more details" "$LINENO" 5; } fi rm -f conftest conftest$ac_cv_exeext { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_exeext" >&5 $as_echo "$ac_cv_exeext" >&6; } rm -f conftest.$ac_ext EXEEXT=$ac_cv_exeext ac_exeext=$EXEEXT cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include int main () { FILE *f = fopen ("conftest.out", "w"); return ferror (f) || fclose (f) != 0; ; return 0; } _ACEOF ac_clean_files="$ac_clean_files conftest.out" # Check that the compiler produces executables we can run. If not, either # the compiler is broken, or we cross compile. { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether we are cross compiling" >&5 $as_echo_n "checking whether we are cross compiling... " >&6; } if test "$cross_compiling" != yes; then { { ac_try="$ac_link" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_link") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } if { ac_try='./conftest$ac_cv_exeext' { { case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_try") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; }; then cross_compiling=no else if test "$cross_compiling" = maybe; then cross_compiling=yes else { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 $as_echo "$as_me: error: in \`$ac_pwd':" >&2;} as_fn_error $? "cannot run C++ compiled programs. If you meant to cross compile, use \`--host'. See \`config.log' for more details" "$LINENO" 5; } fi fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $cross_compiling" >&5 $as_echo "$cross_compiling" >&6; } rm -f conftest.$ac_ext conftest$ac_cv_exeext conftest.out ac_clean_files=$ac_clean_files_save { $as_echo "$as_me:${as_lineno-$LINENO}: checking for suffix of object files" >&5 $as_echo_n "checking for suffix of object files... " >&6; } if ${ac_cv_objext+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF rm -f conftest.o conftest.obj if { { ac_try="$ac_compile" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_compile") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then : for ac_file in conftest.o conftest.obj conftest.*; do test -f "$ac_file" || continue; case $ac_file in *.$ac_ext | *.xcoff | *.tds | *.d | *.pdb | *.xSYM | *.bb | *.bbg | *.map | *.inf | *.dSYM ) ;; *) ac_cv_objext=`expr "$ac_file" : '.*\.\(.*\)'` break;; esac done else $as_echo "$as_me: failed program was:" >&5 sed 's/^/| /' conftest.$ac_ext >&5 { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 $as_echo "$as_me: error: in \`$ac_pwd':" >&2;} as_fn_error $? "cannot compute suffix of object files: cannot compile See \`config.log' for more details" "$LINENO" 5; } fi rm -f conftest.$ac_cv_objext conftest.$ac_ext fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_objext" >&5 $as_echo "$ac_cv_objext" >&6; } OBJEXT=$ac_cv_objext ac_objext=$OBJEXT { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether we are using the GNU C++ compiler" >&5 $as_echo_n "checking whether we are using the GNU C++ compiler... " >&6; } if ${ac_cv_cxx_compiler_gnu+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { #ifndef __GNUC__ choke me #endif ; return 0; } _ACEOF if ac_fn_cxx_try_compile "$LINENO"; then : ac_compiler_gnu=yes else ac_compiler_gnu=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext ac_cv_cxx_compiler_gnu=$ac_compiler_gnu fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_cxx_compiler_gnu" >&5 $as_echo "$ac_cv_cxx_compiler_gnu" >&6; } if test $ac_compiler_gnu = yes; then GXX=yes else GXX= fi ac_test_CXXFLAGS=${CXXFLAGS+set} ac_save_CXXFLAGS=$CXXFLAGS { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether $CXX accepts -g" >&5 $as_echo_n "checking whether $CXX accepts -g... " >&6; } if ${ac_cv_prog_cxx_g+:} false; then : $as_echo_n "(cached) " >&6 else ac_save_cxx_werror_flag=$ac_cxx_werror_flag ac_cxx_werror_flag=yes ac_cv_prog_cxx_g=no CXXFLAGS="-g" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_cxx_try_compile "$LINENO"; then : ac_cv_prog_cxx_g=yes else CXXFLAGS="" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_cxx_try_compile "$LINENO"; then : else ac_cxx_werror_flag=$ac_save_cxx_werror_flag CXXFLAGS="-g" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_cxx_try_compile "$LINENO"; then : ac_cv_prog_cxx_g=yes fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext ac_cxx_werror_flag=$ac_save_cxx_werror_flag fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_prog_cxx_g" >&5 $as_echo "$ac_cv_prog_cxx_g" >&6; } if test "$ac_test_CXXFLAGS" = set; then CXXFLAGS=$ac_save_CXXFLAGS elif test $ac_cv_prog_cxx_g = yes; then if test "$GXX" = yes; then CXXFLAGS="-g -O2" else CXXFLAGS="-g" fi else if test "$GXX" = yes; then CXXFLAGS="-O2" else CXXFLAGS= fi fi ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu DEPDIR="${am__leading_dot}deps" ac_config_commands="$ac_config_commands depfiles" am_make=${MAKE-make} cat > confinc << 'END' am__doit: @echo this is the am__doit target .PHONY: am__doit END # If we don't find an include directive, just comment out the code. { $as_echo "$as_me:${as_lineno-$LINENO}: checking for style of include used by $am_make" >&5 $as_echo_n "checking for style of include used by $am_make... " >&6; } am__include="#" am__quote= _am_result=none # First try GNU make style include. echo "include confinc" > confmf # Ignore all kinds of additional output from 'make'. case `$am_make -s -f confmf 2> /dev/null` in #( *the\ am__doit\ target*) am__include=include am__quote= _am_result=GNU ;; esac # Now try BSD make style include. if test "$am__include" = "#"; then echo '.include "confinc"' > confmf case `$am_make -s -f confmf 2> /dev/null` in #( *the\ am__doit\ target*) am__include=.include am__quote="\"" _am_result=BSD ;; esac fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $_am_result" >&5 $as_echo "$_am_result" >&6; } rm -f confinc confmf # Check whether --enable-dependency-tracking was given. if test "${enable_dependency_tracking+set}" = set; then : enableval=$enable_dependency_tracking; fi if test "x$enable_dependency_tracking" != xno; then am_depcomp="$ac_aux_dir/depcomp" AMDEPBACKSLASH='\' am__nodep='_no' fi if test "x$enable_dependency_tracking" != xno; then AMDEP_TRUE= AMDEP_FALSE='#' else AMDEP_TRUE='#' AMDEP_FALSE= fi depcc="$CXX" am_compiler_list= { $as_echo "$as_me:${as_lineno-$LINENO}: checking dependency style of $depcc" >&5 $as_echo_n "checking dependency style of $depcc... " >&6; } if ${am_cv_CXX_dependencies_compiler_type+:} false; then : $as_echo_n "(cached) " >&6 else if test -z "$AMDEP_TRUE" && test -f "$am_depcomp"; then # We make a subdir and do the tests there. Otherwise we can end up # making bogus files that we don't know about and never remove. For # instance it was reported that on HP-UX the gcc test will end up # making a dummy file named 'D' -- because '-MD' means "put the output # in D". rm -rf conftest.dir mkdir conftest.dir # Copy depcomp to subdir because otherwise we won't find it if we're # using a relative directory. cp "$am_depcomp" conftest.dir cd conftest.dir # We will build objects and dependencies in a subdirectory because # it helps to detect inapplicable dependency modes. For instance # both Tru64's cc and ICC support -MD to output dependencies as a # side effect of compilation, but ICC will put the dependencies in # the current directory while Tru64 will put them in the object # directory. mkdir sub am_cv_CXX_dependencies_compiler_type=none if test "$am_compiler_list" = ""; then am_compiler_list=`sed -n 's/^#*\([a-zA-Z0-9]*\))$/\1/p' < ./depcomp` fi am__universal=false case " $depcc " in #( *\ -arch\ *\ -arch\ *) am__universal=true ;; esac for depmode in $am_compiler_list; do # Setup a source with many dependencies, because some compilers # like to wrap large dependency lists on column 80 (with \), and # we should not choose a depcomp mode which is confused by this. # # We need to recreate these files for each test, as the compiler may # overwrite some of them when testing with obscure command lines. # This happens at least with the AIX C compiler. : > sub/conftest.c for i in 1 2 3 4 5 6; do echo '#include "conftst'$i'.h"' >> sub/conftest.c # Using ": > sub/conftst$i.h" creates only sub/conftst1.h with # Solaris 10 /bin/sh. echo '/* dummy */' > sub/conftst$i.h done echo "${am__include} ${am__quote}sub/conftest.Po${am__quote}" > confmf # We check with '-c' and '-o' for the sake of the "dashmstdout" # mode. It turns out that the SunPro C++ compiler does not properly # handle '-M -o', and we need to detect this. Also, some Intel # versions had trouble with output in subdirs. am__obj=sub/conftest.${OBJEXT-o} am__minus_obj="-o $am__obj" case $depmode in gcc) # This depmode causes a compiler race in universal mode. test "$am__universal" = false || continue ;; nosideeffect) # After this tag, mechanisms are not by side-effect, so they'll # only be used when explicitly requested. if test "x$enable_dependency_tracking" = xyes; then continue else break fi ;; msvc7 | msvc7msys | msvisualcpp | msvcmsys) # This compiler won't grok '-c -o', but also, the minuso test has # not run yet. These depmodes are late enough in the game, and # so weak that their functioning should not be impacted. am__obj=conftest.${OBJEXT-o} am__minus_obj= ;; none) break ;; esac if depmode=$depmode \ source=sub/conftest.c object=$am__obj \ depfile=sub/conftest.Po tmpdepfile=sub/conftest.TPo \ $SHELL ./depcomp $depcc -c $am__minus_obj sub/conftest.c \ >/dev/null 2>conftest.err && grep sub/conftst1.h sub/conftest.Po > /dev/null 2>&1 && grep sub/conftst6.h sub/conftest.Po > /dev/null 2>&1 && grep $am__obj sub/conftest.Po > /dev/null 2>&1 && ${MAKE-make} -s -f confmf > /dev/null 2>&1; then # icc doesn't choke on unknown options, it will just issue warnings # or remarks (even with -Werror). So we grep stderr for any message # that says an option was ignored or not supported. # When given -MP, icc 7.0 and 7.1 complain thusly: # icc: Command line warning: ignoring option '-M'; no argument required # The diagnosis changed in icc 8.0: # icc: Command line remark: option '-MP' not supported if (grep 'ignoring option' conftest.err || grep 'not supported' conftest.err) >/dev/null 2>&1; then :; else am_cv_CXX_dependencies_compiler_type=$depmode break fi fi done cd .. rm -rf conftest.dir else am_cv_CXX_dependencies_compiler_type=none fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $am_cv_CXX_dependencies_compiler_type" >&5 $as_echo "$am_cv_CXX_dependencies_compiler_type" >&6; } CXXDEPMODE=depmode=$am_cv_CXX_dependencies_compiler_type if test "x$enable_dependency_tracking" != xno \ && test "$am_cv_CXX_dependencies_compiler_type" = gcc3; then am__fastdepCXX_TRUE= am__fastdepCXX_FALSE='#' else am__fastdepCXX_TRUE='#' am__fastdepCXX_FALSE= fi ac_ext=cpp ac_cpp='$CXXCPP $CPPFLAGS' ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_cxx_compiler_gnu { $as_echo "$as_me:${as_lineno-$LINENO}: checking how to run the C++ preprocessor" >&5 $as_echo_n "checking how to run the C++ preprocessor... " >&6; } if test -z "$CXXCPP"; then if ${ac_cv_prog_CXXCPP+:} false; then : $as_echo_n "(cached) " >&6 else # Double quotes because CXXCPP needs to be expanded for CXXCPP in "$CXX -E" "/lib/cpp" do ac_preproc_ok=false for ac_cxx_preproc_warn_flag in '' yes do # Use a header file that comes with gcc, so configuring glibc # with a fresh cross-compiler works. # Prefer to if __STDC__ is defined, since # exists even on freestanding compilers. # On the NeXT, cc -E runs the code through the compiler's parser, # not just through cpp. "Syntax error" is here to catch this case. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #ifdef __STDC__ # include #else # include #endif Syntax error _ACEOF if ac_fn_cxx_try_cpp "$LINENO"; then : else # Broken: fails on valid input. continue fi rm -f conftest.err conftest.i conftest.$ac_ext # OK, works on sane cases. Now check whether nonexistent headers # can be detected and how. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include _ACEOF if ac_fn_cxx_try_cpp "$LINENO"; then : # Broken: success on invalid input. continue else # Passes both tests. ac_preproc_ok=: break fi rm -f conftest.err conftest.i conftest.$ac_ext done # Because of `break', _AC_PREPROC_IFELSE's cleaning code was skipped. rm -f conftest.i conftest.err conftest.$ac_ext if $ac_preproc_ok; then : break fi done ac_cv_prog_CXXCPP=$CXXCPP fi CXXCPP=$ac_cv_prog_CXXCPP else ac_cv_prog_CXXCPP=$CXXCPP fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $CXXCPP" >&5 $as_echo "$CXXCPP" >&6; } ac_preproc_ok=false for ac_cxx_preproc_warn_flag in '' yes do # Use a header file that comes with gcc, so configuring glibc # with a fresh cross-compiler works. # Prefer to if __STDC__ is defined, since # exists even on freestanding compilers. # On the NeXT, cc -E runs the code through the compiler's parser, # not just through cpp. "Syntax error" is here to catch this case. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #ifdef __STDC__ # include #else # include #endif Syntax error _ACEOF if ac_fn_cxx_try_cpp "$LINENO"; then : else # Broken: fails on valid input. continue fi rm -f conftest.err conftest.i conftest.$ac_ext # OK, works on sane cases. Now check whether nonexistent headers # can be detected and how. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include _ACEOF if ac_fn_cxx_try_cpp "$LINENO"; then : # Broken: success on invalid input. continue else # Passes both tests. ac_preproc_ok=: break fi rm -f conftest.err conftest.i conftest.$ac_ext done # Because of `break', _AC_PREPROC_IFELSE's cleaning code was skipped. rm -f conftest.i conftest.err conftest.$ac_ext if $ac_preproc_ok; then : else { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 $as_echo "$as_me: error: in \`$ac_pwd':" >&2;} as_fn_error $? "C++ preprocessor \"$CXXCPP\" fails sanity check See \`config.log' for more details" "$LINENO" 5; } fi ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu ac_ext=cpp ac_cpp='$CXXCPP $CPPFLAGS' ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_cxx_compiler_gnu # Make sure we can run config.sub. $SHELL "$ac_aux_dir/config.sub" sun4 >/dev/null 2>&1 || as_fn_error $? "cannot run $SHELL $ac_aux_dir/config.sub" "$LINENO" 5 { $as_echo "$as_me:${as_lineno-$LINENO}: checking build system type" >&5 $as_echo_n "checking build system type... " >&6; } if ${ac_cv_build+:} false; then : $as_echo_n "(cached) " >&6 else ac_build_alias=$build_alias test "x$ac_build_alias" = x && ac_build_alias=`$SHELL "$ac_aux_dir/config.guess"` test "x$ac_build_alias" = x && as_fn_error $? "cannot guess build type; you must specify one" "$LINENO" 5 ac_cv_build=`$SHELL "$ac_aux_dir/config.sub" $ac_build_alias` || as_fn_error $? "$SHELL $ac_aux_dir/config.sub $ac_build_alias failed" "$LINENO" 5 fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_build" >&5 $as_echo "$ac_cv_build" >&6; } case $ac_cv_build in *-*-*) ;; *) as_fn_error $? "invalid value of canonical build" "$LINENO" 5;; esac build=$ac_cv_build ac_save_IFS=$IFS; IFS='-' set x $ac_cv_build shift build_cpu=$1 build_vendor=$2 shift; shift # Remember, the first character of IFS is used to create $*, # except with old shells: build_os=$* IFS=$ac_save_IFS case $build_os in *\ *) build_os=`echo "$build_os" | sed 's/ /-/g'`;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: checking host system type" >&5 $as_echo_n "checking host system type... " >&6; } if ${ac_cv_host+:} false; then : $as_echo_n "(cached) " >&6 else if test "x$host_alias" = x; then ac_cv_host=$ac_cv_build else ac_cv_host=`$SHELL "$ac_aux_dir/config.sub" $host_alias` || as_fn_error $? "$SHELL $ac_aux_dir/config.sub $host_alias failed" "$LINENO" 5 fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_host" >&5 $as_echo "$ac_cv_host" >&6; } case $ac_cv_host in *-*-*) ;; *) as_fn_error $? "invalid value of canonical host" "$LINENO" 5;; esac host=$ac_cv_host ac_save_IFS=$IFS; IFS='-' set x $ac_cv_host shift host_cpu=$1 host_vendor=$2 shift; shift # Remember, the first character of IFS is used to create $*, # except with old shells: host_os=$* IFS=$ac_save_IFS case $host_os in *\ *) host_os=`echo "$host_os" | sed 's/ /-/g'`;; esac case `pwd` in *\ * | *\ *) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: Libtool does not cope well with whitespace in \`pwd\`" >&5 $as_echo "$as_me: WARNING: Libtool does not cope well with whitespace in \`pwd\`" >&2;} ;; esac macro_version='2.4.6' macro_revision='2.4.6' ltmain=$ac_aux_dir/ltmain.sh # Backslashify metacharacters that are still active within # double-quoted strings. sed_quote_subst='s/\(["`$\\]\)/\\\1/g' # Same as above, but do not quote variable references. double_quote_subst='s/\(["`\\]\)/\\\1/g' # Sed substitution to delay expansion of an escaped shell variable in a # double_quote_subst'ed string. delay_variable_subst='s/\\\\\\\\\\\$/\\\\\\$/g' # Sed substitution to delay expansion of an escaped single quote. delay_single_quote_subst='s/'\''/'\'\\\\\\\'\''/g' # Sed substitution to avoid accidental globbing in evaled expressions no_glob_subst='s/\*/\\\*/g' ECHO='\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\' ECHO=$ECHO$ECHO$ECHO$ECHO$ECHO ECHO=$ECHO$ECHO$ECHO$ECHO$ECHO$ECHO { $as_echo "$as_me:${as_lineno-$LINENO}: checking how to print strings" >&5 $as_echo_n "checking how to print strings... " >&6; } # Test print first, because it will be a builtin if present. if test "X`( print -r -- -n ) 2>/dev/null`" = X-n && \ test "X`print -r -- $ECHO 2>/dev/null`" = "X$ECHO"; then ECHO='print -r --' elif test "X`printf %s $ECHO 2>/dev/null`" = "X$ECHO"; then ECHO='printf %s\n' else # Use this function as a fallback that always works. func_fallback_echo () { eval 'cat <<_LTECHO_EOF $1 _LTECHO_EOF' } ECHO='func_fallback_echo' fi # func_echo_all arg... # Invoke $ECHO with all args, space-separated. func_echo_all () { $ECHO "" } case $ECHO in printf*) { $as_echo "$as_me:${as_lineno-$LINENO}: result: printf" >&5 $as_echo "printf" >&6; } ;; print*) { $as_echo "$as_me:${as_lineno-$LINENO}: result: print -r" >&5 $as_echo "print -r" >&6; } ;; *) { $as_echo "$as_me:${as_lineno-$LINENO}: result: cat" >&5 $as_echo "cat" >&6; } ;; esac ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}gcc", so it can be a program name with args. set dummy ${ac_tool_prefix}gcc; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_CC+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$CC"; then ac_cv_prog_CC="$CC" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_CC="${ac_tool_prefix}gcc" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi CC=$ac_cv_prog_CC if test -n "$CC"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $CC" >&5 $as_echo "$CC" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_CC"; then ac_ct_CC=$CC # Extract the first word of "gcc", so it can be a program name with args. set dummy gcc; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_CC+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_CC"; then ac_cv_prog_ac_ct_CC="$ac_ct_CC" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_CC="gcc" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_CC=$ac_cv_prog_ac_ct_CC if test -n "$ac_ct_CC"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_CC" >&5 $as_echo "$ac_ct_CC" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_CC" = x; then CC="" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac CC=$ac_ct_CC fi else CC="$ac_cv_prog_CC" fi if test -z "$CC"; then if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}cc", so it can be a program name with args. set dummy ${ac_tool_prefix}cc; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_CC+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$CC"; then ac_cv_prog_CC="$CC" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_CC="${ac_tool_prefix}cc" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi CC=$ac_cv_prog_CC if test -n "$CC"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $CC" >&5 $as_echo "$CC" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi fi if test -z "$CC"; then # Extract the first word of "cc", so it can be a program name with args. set dummy cc; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_CC+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$CC"; then ac_cv_prog_CC="$CC" # Let the user override the test. else ac_prog_rejected=no as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then if test "$as_dir/$ac_word$ac_exec_ext" = "/usr/ucb/cc"; then ac_prog_rejected=yes continue fi ac_cv_prog_CC="cc" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS if test $ac_prog_rejected = yes; then # We found a bogon in the path, so make sure we never use it. set dummy $ac_cv_prog_CC shift if test $# != 0; then # We chose a different compiler from the bogus one. # However, it has the same basename, so the bogon will be chosen # first if we set CC to just the basename; use the full file name. shift ac_cv_prog_CC="$as_dir/$ac_word${1+' '}$@" fi fi fi fi CC=$ac_cv_prog_CC if test -n "$CC"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $CC" >&5 $as_echo "$CC" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$CC"; then if test -n "$ac_tool_prefix"; then for ac_prog in cl.exe do # Extract the first word of "$ac_tool_prefix$ac_prog", so it can be a program name with args. set dummy $ac_tool_prefix$ac_prog; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_CC+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$CC"; then ac_cv_prog_CC="$CC" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_CC="$ac_tool_prefix$ac_prog" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi CC=$ac_cv_prog_CC if test -n "$CC"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $CC" >&5 $as_echo "$CC" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi test -n "$CC" && break done fi if test -z "$CC"; then ac_ct_CC=$CC for ac_prog in cl.exe do # Extract the first word of "$ac_prog", so it can be a program name with args. set dummy $ac_prog; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_CC+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_CC"; then ac_cv_prog_ac_ct_CC="$ac_ct_CC" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_CC="$ac_prog" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_CC=$ac_cv_prog_ac_ct_CC if test -n "$ac_ct_CC"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_CC" >&5 $as_echo "$ac_ct_CC" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi test -n "$ac_ct_CC" && break done if test "x$ac_ct_CC" = x; then CC="" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac CC=$ac_ct_CC fi fi fi test -z "$CC" && { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 $as_echo "$as_me: error: in \`$ac_pwd':" >&2;} as_fn_error $? "no acceptable C compiler found in \$PATH See \`config.log' for more details" "$LINENO" 5; } # Provide some information about the compiler. $as_echo "$as_me:${as_lineno-$LINENO}: checking for C compiler version" >&5 set X $ac_compile ac_compiler=$2 for ac_option in --version -v -V -qversion; do { { ac_try="$ac_compiler $ac_option >&5" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_compiler $ac_option >&5") 2>conftest.err ac_status=$? if test -s conftest.err; then sed '10a\ ... rest of stderr output deleted ... 10q' conftest.err >conftest.er1 cat conftest.er1 >&5 fi rm -f conftest.er1 conftest.err $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } done { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether we are using the GNU C compiler" >&5 $as_echo_n "checking whether we are using the GNU C compiler... " >&6; } if ${ac_cv_c_compiler_gnu+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { #ifndef __GNUC__ choke me #endif ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_compiler_gnu=yes else ac_compiler_gnu=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext ac_cv_c_compiler_gnu=$ac_compiler_gnu fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_c_compiler_gnu" >&5 $as_echo "$ac_cv_c_compiler_gnu" >&6; } if test $ac_compiler_gnu = yes; then GCC=yes else GCC= fi ac_test_CFLAGS=${CFLAGS+set} ac_save_CFLAGS=$CFLAGS { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether $CC accepts -g" >&5 $as_echo_n "checking whether $CC accepts -g... " >&6; } if ${ac_cv_prog_cc_g+:} false; then : $as_echo_n "(cached) " >&6 else ac_save_c_werror_flag=$ac_c_werror_flag ac_c_werror_flag=yes ac_cv_prog_cc_g=no CFLAGS="-g" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_cv_prog_cc_g=yes else CFLAGS="" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : else ac_c_werror_flag=$ac_save_c_werror_flag CFLAGS="-g" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_cv_prog_cc_g=yes fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext ac_c_werror_flag=$ac_save_c_werror_flag fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_prog_cc_g" >&5 $as_echo "$ac_cv_prog_cc_g" >&6; } if test "$ac_test_CFLAGS" = set; then CFLAGS=$ac_save_CFLAGS elif test $ac_cv_prog_cc_g = yes; then if test "$GCC" = yes; then CFLAGS="-g -O2" else CFLAGS="-g" fi else if test "$GCC" = yes; then CFLAGS="-O2" else CFLAGS= fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $CC option to accept ISO C89" >&5 $as_echo_n "checking for $CC option to accept ISO C89... " >&6; } if ${ac_cv_prog_cc_c89+:} false; then : $as_echo_n "(cached) " >&6 else ac_cv_prog_cc_c89=no ac_save_CC=$CC cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include #include struct stat; /* Most of the following tests are stolen from RCS 5.7's src/conf.sh. */ struct buf { int x; }; FILE * (*rcsopen) (struct buf *, struct stat *, int); static char *e (p, i) char **p; int i; { return p[i]; } static char *f (char * (*g) (char **, int), char **p, ...) { char *s; va_list v; va_start (v,p); s = g (p, va_arg (v,int)); va_end (v); return s; } /* OSF 4.0 Compaq cc is some sort of almost-ANSI by default. It has function prototypes and stuff, but not '\xHH' hex character constants. These don't provoke an error unfortunately, instead are silently treated as 'x'. The following induces an error, until -std is added to get proper ANSI mode. Curiously '\x00'!='x' always comes out true, for an array size at least. It's necessary to write '\x00'==0 to get something that's true only with -std. */ int osf4_cc_array ['\x00' == 0 ? 1 : -1]; /* IBM C 6 for AIX is almost-ANSI by default, but it replaces macro parameters inside strings and character constants. */ #define FOO(x) 'x' int xlc6_cc_array[FOO(a) == 'x' ? 1 : -1]; int test (int i, double x); struct s1 {int (*f) (int a);}; struct s2 {int (*f) (double a);}; int pairnames (int, char **, FILE *(*)(struct buf *, struct stat *, int), int, int); int argc; char **argv; int main () { return f (e, argv, 0) != argv[0] || f (e, argv, 1) != argv[1]; ; return 0; } _ACEOF for ac_arg in '' -qlanglvl=extc89 -qlanglvl=ansi -std \ -Ae "-Aa -D_HPUX_SOURCE" "-Xc -D__EXTENSIONS__" do CC="$ac_save_CC $ac_arg" if ac_fn_c_try_compile "$LINENO"; then : ac_cv_prog_cc_c89=$ac_arg fi rm -f core conftest.err conftest.$ac_objext test "x$ac_cv_prog_cc_c89" != "xno" && break done rm -f conftest.$ac_ext CC=$ac_save_CC fi # AC_CACHE_VAL case "x$ac_cv_prog_cc_c89" in x) { $as_echo "$as_me:${as_lineno-$LINENO}: result: none needed" >&5 $as_echo "none needed" >&6; } ;; xno) { $as_echo "$as_me:${as_lineno-$LINENO}: result: unsupported" >&5 $as_echo "unsupported" >&6; } ;; *) CC="$CC $ac_cv_prog_cc_c89" { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_prog_cc_c89" >&5 $as_echo "$ac_cv_prog_cc_c89" >&6; } ;; esac if test "x$ac_cv_prog_cc_c89" != xno; then : fi ac_ext=cpp ac_cpp='$CXXCPP $CPPFLAGS' ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_cxx_compiler_gnu ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether $CC understands -c and -o together" >&5 $as_echo_n "checking whether $CC understands -c and -o together... " >&6; } if ${am_cv_prog_cc_c_o+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF # Make sure it works both with $CC and with simple cc. # Following AC_PROG_CC_C_O, we do the test twice because some # compilers refuse to overwrite an existing .o file with -o, # though they will create one. am_cv_prog_cc_c_o=yes for am_i in 1 2; do if { echo "$as_me:$LINENO: $CC -c conftest.$ac_ext -o conftest2.$ac_objext" >&5 ($CC -c conftest.$ac_ext -o conftest2.$ac_objext) >&5 2>&5 ac_status=$? echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } \ && test -f conftest2.$ac_objext; then : OK else am_cv_prog_cc_c_o=no break fi done rm -f core conftest* unset am_i fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $am_cv_prog_cc_c_o" >&5 $as_echo "$am_cv_prog_cc_c_o" >&6; } if test "$am_cv_prog_cc_c_o" != yes; then # Losing compiler, so override with the script. # FIXME: It is wrong to rewrite CC. # But if we don't then we get into trouble of one sort or another. # A longer-term fix would be to have automake use am__CC in this case, # and then we could set am__CC="\$(top_srcdir)/compile \$(CC)" CC="$am_aux_dir/compile $CC" fi ac_ext=cpp ac_cpp='$CXXCPP $CPPFLAGS' ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_cxx_compiler_gnu depcc="$CC" am_compiler_list= { $as_echo "$as_me:${as_lineno-$LINENO}: checking dependency style of $depcc" >&5 $as_echo_n "checking dependency style of $depcc... " >&6; } if ${am_cv_CC_dependencies_compiler_type+:} false; then : $as_echo_n "(cached) " >&6 else if test -z "$AMDEP_TRUE" && test -f "$am_depcomp"; then # We make a subdir and do the tests there. Otherwise we can end up # making bogus files that we don't know about and never remove. For # instance it was reported that on HP-UX the gcc test will end up # making a dummy file named 'D' -- because '-MD' means "put the output # in D". rm -rf conftest.dir mkdir conftest.dir # Copy depcomp to subdir because otherwise we won't find it if we're # using a relative directory. cp "$am_depcomp" conftest.dir cd conftest.dir # We will build objects and dependencies in a subdirectory because # it helps to detect inapplicable dependency modes. For instance # both Tru64's cc and ICC support -MD to output dependencies as a # side effect of compilation, but ICC will put the dependencies in # the current directory while Tru64 will put them in the object # directory. mkdir sub am_cv_CC_dependencies_compiler_type=none if test "$am_compiler_list" = ""; then am_compiler_list=`sed -n 's/^#*\([a-zA-Z0-9]*\))$/\1/p' < ./depcomp` fi am__universal=false case " $depcc " in #( *\ -arch\ *\ -arch\ *) am__universal=true ;; esac for depmode in $am_compiler_list; do # Setup a source with many dependencies, because some compilers # like to wrap large dependency lists on column 80 (with \), and # we should not choose a depcomp mode which is confused by this. # # We need to recreate these files for each test, as the compiler may # overwrite some of them when testing with obscure command lines. # This happens at least with the AIX C compiler. : > sub/conftest.c for i in 1 2 3 4 5 6; do echo '#include "conftst'$i'.h"' >> sub/conftest.c # Using ": > sub/conftst$i.h" creates only sub/conftst1.h with # Solaris 10 /bin/sh. echo '/* dummy */' > sub/conftst$i.h done echo "${am__include} ${am__quote}sub/conftest.Po${am__quote}" > confmf # We check with '-c' and '-o' for the sake of the "dashmstdout" # mode. It turns out that the SunPro C++ compiler does not properly # handle '-M -o', and we need to detect this. Also, some Intel # versions had trouble with output in subdirs. am__obj=sub/conftest.${OBJEXT-o} am__minus_obj="-o $am__obj" case $depmode in gcc) # This depmode causes a compiler race in universal mode. test "$am__universal" = false || continue ;; nosideeffect) # After this tag, mechanisms are not by side-effect, so they'll # only be used when explicitly requested. if test "x$enable_dependency_tracking" = xyes; then continue else break fi ;; msvc7 | msvc7msys | msvisualcpp | msvcmsys) # This compiler won't grok '-c -o', but also, the minuso test has # not run yet. These depmodes are late enough in the game, and # so weak that their functioning should not be impacted. am__obj=conftest.${OBJEXT-o} am__minus_obj= ;; none) break ;; esac if depmode=$depmode \ source=sub/conftest.c object=$am__obj \ depfile=sub/conftest.Po tmpdepfile=sub/conftest.TPo \ $SHELL ./depcomp $depcc -c $am__minus_obj sub/conftest.c \ >/dev/null 2>conftest.err && grep sub/conftst1.h sub/conftest.Po > /dev/null 2>&1 && grep sub/conftst6.h sub/conftest.Po > /dev/null 2>&1 && grep $am__obj sub/conftest.Po > /dev/null 2>&1 && ${MAKE-make} -s -f confmf > /dev/null 2>&1; then # icc doesn't choke on unknown options, it will just issue warnings # or remarks (even with -Werror). So we grep stderr for any message # that says an option was ignored or not supported. # When given -MP, icc 7.0 and 7.1 complain thusly: # icc: Command line warning: ignoring option '-M'; no argument required # The diagnosis changed in icc 8.0: # icc: Command line remark: option '-MP' not supported if (grep 'ignoring option' conftest.err || grep 'not supported' conftest.err) >/dev/null 2>&1; then :; else am_cv_CC_dependencies_compiler_type=$depmode break fi fi done cd .. rm -rf conftest.dir else am_cv_CC_dependencies_compiler_type=none fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $am_cv_CC_dependencies_compiler_type" >&5 $as_echo "$am_cv_CC_dependencies_compiler_type" >&6; } CCDEPMODE=depmode=$am_cv_CC_dependencies_compiler_type if test "x$enable_dependency_tracking" != xno \ && test "$am_cv_CC_dependencies_compiler_type" = gcc3; then am__fastdepCC_TRUE= am__fastdepCC_FALSE='#' else am__fastdepCC_TRUE='#' am__fastdepCC_FALSE= fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for a sed that does not truncate output" >&5 $as_echo_n "checking for a sed that does not truncate output... " >&6; } if ${ac_cv_path_SED+:} false; then : $as_echo_n "(cached) " >&6 else ac_script=s/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb/ for ac_i in 1 2 3 4 5 6 7; do ac_script="$ac_script$as_nl$ac_script" done echo "$ac_script" 2>/dev/null | sed 99q >conftest.sed { ac_script=; unset ac_script;} if test -z "$SED"; then ac_path_SED_found=false # Loop through the user's path and test for each of PROGNAME-LIST as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_prog in sed gsed; do for ac_exec_ext in '' $ac_executable_extensions; do ac_path_SED="$as_dir/$ac_prog$ac_exec_ext" as_fn_executable_p "$ac_path_SED" || continue # Check for GNU ac_path_SED and select it if it is found. # Check for GNU $ac_path_SED case `"$ac_path_SED" --version 2>&1` in *GNU*) ac_cv_path_SED="$ac_path_SED" ac_path_SED_found=:;; *) ac_count=0 $as_echo_n 0123456789 >"conftest.in" while : do cat "conftest.in" "conftest.in" >"conftest.tmp" mv "conftest.tmp" "conftest.in" cp "conftest.in" "conftest.nl" $as_echo '' >> "conftest.nl" "$ac_path_SED" -f conftest.sed < "conftest.nl" >"conftest.out" 2>/dev/null || break diff "conftest.out" "conftest.nl" >/dev/null 2>&1 || break as_fn_arith $ac_count + 1 && ac_count=$as_val if test $ac_count -gt ${ac_path_SED_max-0}; then # Best one so far, save it but keep looking for a better one ac_cv_path_SED="$ac_path_SED" ac_path_SED_max=$ac_count fi # 10*(2^10) chars as input seems more than enough test $ac_count -gt 10 && break done rm -f conftest.in conftest.tmp conftest.nl conftest.out;; esac $ac_path_SED_found && break 3 done done done IFS=$as_save_IFS if test -z "$ac_cv_path_SED"; then as_fn_error $? "no acceptable sed could be found in \$PATH" "$LINENO" 5 fi else ac_cv_path_SED=$SED fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_path_SED" >&5 $as_echo "$ac_cv_path_SED" >&6; } SED="$ac_cv_path_SED" rm -f conftest.sed test -z "$SED" && SED=sed Xsed="$SED -e 1s/^X//" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for grep that handles long lines and -e" >&5 $as_echo_n "checking for grep that handles long lines and -e... " >&6; } if ${ac_cv_path_GREP+:} false; then : $as_echo_n "(cached) " >&6 else if test -z "$GREP"; then ac_path_GREP_found=false # Loop through the user's path and test for each of PROGNAME-LIST as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH$PATH_SEPARATOR/usr/xpg4/bin do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_prog in grep ggrep; do for ac_exec_ext in '' $ac_executable_extensions; do ac_path_GREP="$as_dir/$ac_prog$ac_exec_ext" as_fn_executable_p "$ac_path_GREP" || continue # Check for GNU ac_path_GREP and select it if it is found. # Check for GNU $ac_path_GREP case `"$ac_path_GREP" --version 2>&1` in *GNU*) ac_cv_path_GREP="$ac_path_GREP" ac_path_GREP_found=:;; *) ac_count=0 $as_echo_n 0123456789 >"conftest.in" while : do cat "conftest.in" "conftest.in" >"conftest.tmp" mv "conftest.tmp" "conftest.in" cp "conftest.in" "conftest.nl" $as_echo 'GREP' >> "conftest.nl" "$ac_path_GREP" -e 'GREP$' -e '-(cannot match)-' < "conftest.nl" >"conftest.out" 2>/dev/null || break diff "conftest.out" "conftest.nl" >/dev/null 2>&1 || break as_fn_arith $ac_count + 1 && ac_count=$as_val if test $ac_count -gt ${ac_path_GREP_max-0}; then # Best one so far, save it but keep looking for a better one ac_cv_path_GREP="$ac_path_GREP" ac_path_GREP_max=$ac_count fi # 10*(2^10) chars as input seems more than enough test $ac_count -gt 10 && break done rm -f conftest.in conftest.tmp conftest.nl conftest.out;; esac $ac_path_GREP_found && break 3 done done done IFS=$as_save_IFS if test -z "$ac_cv_path_GREP"; then as_fn_error $? "no acceptable grep could be found in $PATH$PATH_SEPARATOR/usr/xpg4/bin" "$LINENO" 5 fi else ac_cv_path_GREP=$GREP fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_path_GREP" >&5 $as_echo "$ac_cv_path_GREP" >&6; } GREP="$ac_cv_path_GREP" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for egrep" >&5 $as_echo_n "checking for egrep... " >&6; } if ${ac_cv_path_EGREP+:} false; then : $as_echo_n "(cached) " >&6 else if echo a | $GREP -E '(a|b)' >/dev/null 2>&1 then ac_cv_path_EGREP="$GREP -E" else if test -z "$EGREP"; then ac_path_EGREP_found=false # Loop through the user's path and test for each of PROGNAME-LIST as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH$PATH_SEPARATOR/usr/xpg4/bin do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_prog in egrep; do for ac_exec_ext in '' $ac_executable_extensions; do ac_path_EGREP="$as_dir/$ac_prog$ac_exec_ext" as_fn_executable_p "$ac_path_EGREP" || continue # Check for GNU ac_path_EGREP and select it if it is found. # Check for GNU $ac_path_EGREP case `"$ac_path_EGREP" --version 2>&1` in *GNU*) ac_cv_path_EGREP="$ac_path_EGREP" ac_path_EGREP_found=:;; *) ac_count=0 $as_echo_n 0123456789 >"conftest.in" while : do cat "conftest.in" "conftest.in" >"conftest.tmp" mv "conftest.tmp" "conftest.in" cp "conftest.in" "conftest.nl" $as_echo 'EGREP' >> "conftest.nl" "$ac_path_EGREP" 'EGREP$' < "conftest.nl" >"conftest.out" 2>/dev/null || break diff "conftest.out" "conftest.nl" >/dev/null 2>&1 || break as_fn_arith $ac_count + 1 && ac_count=$as_val if test $ac_count -gt ${ac_path_EGREP_max-0}; then # Best one so far, save it but keep looking for a better one ac_cv_path_EGREP="$ac_path_EGREP" ac_path_EGREP_max=$ac_count fi # 10*(2^10) chars as input seems more than enough test $ac_count -gt 10 && break done rm -f conftest.in conftest.tmp conftest.nl conftest.out;; esac $ac_path_EGREP_found && break 3 done done done IFS=$as_save_IFS if test -z "$ac_cv_path_EGREP"; then as_fn_error $? "no acceptable egrep could be found in $PATH$PATH_SEPARATOR/usr/xpg4/bin" "$LINENO" 5 fi else ac_cv_path_EGREP=$EGREP fi fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_path_EGREP" >&5 $as_echo "$ac_cv_path_EGREP" >&6; } EGREP="$ac_cv_path_EGREP" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for fgrep" >&5 $as_echo_n "checking for fgrep... " >&6; } if ${ac_cv_path_FGREP+:} false; then : $as_echo_n "(cached) " >&6 else if echo 'ab*c' | $GREP -F 'ab*c' >/dev/null 2>&1 then ac_cv_path_FGREP="$GREP -F" else if test -z "$FGREP"; then ac_path_FGREP_found=false # Loop through the user's path and test for each of PROGNAME-LIST as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH$PATH_SEPARATOR/usr/xpg4/bin do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_prog in fgrep; do for ac_exec_ext in '' $ac_executable_extensions; do ac_path_FGREP="$as_dir/$ac_prog$ac_exec_ext" as_fn_executable_p "$ac_path_FGREP" || continue # Check for GNU ac_path_FGREP and select it if it is found. # Check for GNU $ac_path_FGREP case `"$ac_path_FGREP" --version 2>&1` in *GNU*) ac_cv_path_FGREP="$ac_path_FGREP" ac_path_FGREP_found=:;; *) ac_count=0 $as_echo_n 0123456789 >"conftest.in" while : do cat "conftest.in" "conftest.in" >"conftest.tmp" mv "conftest.tmp" "conftest.in" cp "conftest.in" "conftest.nl" $as_echo 'FGREP' >> "conftest.nl" "$ac_path_FGREP" FGREP < "conftest.nl" >"conftest.out" 2>/dev/null || break diff "conftest.out" "conftest.nl" >/dev/null 2>&1 || break as_fn_arith $ac_count + 1 && ac_count=$as_val if test $ac_count -gt ${ac_path_FGREP_max-0}; then # Best one so far, save it but keep looking for a better one ac_cv_path_FGREP="$ac_path_FGREP" ac_path_FGREP_max=$ac_count fi # 10*(2^10) chars as input seems more than enough test $ac_count -gt 10 && break done rm -f conftest.in conftest.tmp conftest.nl conftest.out;; esac $ac_path_FGREP_found && break 3 done done done IFS=$as_save_IFS if test -z "$ac_cv_path_FGREP"; then as_fn_error $? "no acceptable fgrep could be found in $PATH$PATH_SEPARATOR/usr/xpg4/bin" "$LINENO" 5 fi else ac_cv_path_FGREP=$FGREP fi fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_path_FGREP" >&5 $as_echo "$ac_cv_path_FGREP" >&6; } FGREP="$ac_cv_path_FGREP" test -z "$GREP" && GREP=grep # Check whether --with-gnu-ld was given. if test "${with_gnu_ld+set}" = set; then : withval=$with_gnu_ld; test no = "$withval" || with_gnu_ld=yes else with_gnu_ld=no fi ac_prog=ld if test yes = "$GCC"; then # Check if gcc -print-prog-name=ld gives a path. { $as_echo "$as_me:${as_lineno-$LINENO}: checking for ld used by $CC" >&5 $as_echo_n "checking for ld used by $CC... " >&6; } case $host in *-*-mingw*) # gcc leaves a trailing carriage return, which upsets mingw ac_prog=`($CC -print-prog-name=ld) 2>&5 | tr -d '\015'` ;; *) ac_prog=`($CC -print-prog-name=ld) 2>&5` ;; esac case $ac_prog in # Accept absolute paths. [\\/]* | ?:[\\/]*) re_direlt='/[^/][^/]*/\.\./' # Canonicalize the pathname of ld ac_prog=`$ECHO "$ac_prog"| $SED 's%\\\\%/%g'` while $ECHO "$ac_prog" | $GREP "$re_direlt" > /dev/null 2>&1; do ac_prog=`$ECHO $ac_prog| $SED "s%$re_direlt%/%"` done test -z "$LD" && LD=$ac_prog ;; "") # If it fails, then pretend we aren't using GCC. ac_prog=ld ;; *) # If it is relative, then search for the first ld in PATH. with_gnu_ld=unknown ;; esac elif test yes = "$with_gnu_ld"; then { $as_echo "$as_me:${as_lineno-$LINENO}: checking for GNU ld" >&5 $as_echo_n "checking for GNU ld... " >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: checking for non-GNU ld" >&5 $as_echo_n "checking for non-GNU ld... " >&6; } fi if ${lt_cv_path_LD+:} false; then : $as_echo_n "(cached) " >&6 else if test -z "$LD"; then lt_save_ifs=$IFS; IFS=$PATH_SEPARATOR for ac_dir in $PATH; do IFS=$lt_save_ifs test -z "$ac_dir" && ac_dir=. if test -f "$ac_dir/$ac_prog" || test -f "$ac_dir/$ac_prog$ac_exeext"; then lt_cv_path_LD=$ac_dir/$ac_prog # Check to see if the program is GNU ld. I'd rather use --version, # but apparently some variants of GNU ld only accept -v. # Break only if it was the GNU/non-GNU ld that we prefer. case `"$lt_cv_path_LD" -v 2>&1 &5 $as_echo "$LD" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi test -z "$LD" && as_fn_error $? "no acceptable ld found in \$PATH" "$LINENO" 5 { $as_echo "$as_me:${as_lineno-$LINENO}: checking if the linker ($LD) is GNU ld" >&5 $as_echo_n "checking if the linker ($LD) is GNU ld... " >&6; } if ${lt_cv_prog_gnu_ld+:} false; then : $as_echo_n "(cached) " >&6 else # I'd rather use --version here, but apparently some GNU lds only accept -v. case `$LD -v 2>&1 &5 $as_echo "$lt_cv_prog_gnu_ld" >&6; } with_gnu_ld=$lt_cv_prog_gnu_ld { $as_echo "$as_me:${as_lineno-$LINENO}: checking for BSD- or MS-compatible name lister (nm)" >&5 $as_echo_n "checking for BSD- or MS-compatible name lister (nm)... " >&6; } if ${lt_cv_path_NM+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$NM"; then # Let the user override the test. lt_cv_path_NM=$NM else lt_nm_to_check=${ac_tool_prefix}nm if test -n "$ac_tool_prefix" && test "$build" = "$host"; then lt_nm_to_check="$lt_nm_to_check nm" fi for lt_tmp_nm in $lt_nm_to_check; do lt_save_ifs=$IFS; IFS=$PATH_SEPARATOR for ac_dir in $PATH /usr/ccs/bin/elf /usr/ccs/bin /usr/ucb /bin; do IFS=$lt_save_ifs test -z "$ac_dir" && ac_dir=. tmp_nm=$ac_dir/$lt_tmp_nm if test -f "$tmp_nm" || test -f "$tmp_nm$ac_exeext"; then # Check to see if the nm accepts a BSD-compat flag. # Adding the 'sed 1q' prevents false positives on HP-UX, which says: # nm: unknown option "B" ignored # Tru64's nm complains that /dev/null is an invalid object file # MSYS converts /dev/null to NUL, MinGW nm treats NUL as empty case $build_os in mingw*) lt_bad_file=conftest.nm/nofile ;; *) lt_bad_file=/dev/null ;; esac case `"$tmp_nm" -B $lt_bad_file 2>&1 | sed '1q'` in *$lt_bad_file* | *'Invalid file or object type'*) lt_cv_path_NM="$tmp_nm -B" break 2 ;; *) case `"$tmp_nm" -p /dev/null 2>&1 | sed '1q'` in */dev/null*) lt_cv_path_NM="$tmp_nm -p" break 2 ;; *) lt_cv_path_NM=${lt_cv_path_NM="$tmp_nm"} # keep the first match, but continue # so that we can try to find one that supports BSD flags ;; esac ;; esac fi done IFS=$lt_save_ifs done : ${lt_cv_path_NM=no} fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_path_NM" >&5 $as_echo "$lt_cv_path_NM" >&6; } if test no != "$lt_cv_path_NM"; then NM=$lt_cv_path_NM else # Didn't find any BSD compatible name lister, look for dumpbin. if test -n "$DUMPBIN"; then : # Let the user override the test. else if test -n "$ac_tool_prefix"; then for ac_prog in dumpbin "link -dump" do # Extract the first word of "$ac_tool_prefix$ac_prog", so it can be a program name with args. set dummy $ac_tool_prefix$ac_prog; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_DUMPBIN+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$DUMPBIN"; then ac_cv_prog_DUMPBIN="$DUMPBIN" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_DUMPBIN="$ac_tool_prefix$ac_prog" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi DUMPBIN=$ac_cv_prog_DUMPBIN if test -n "$DUMPBIN"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $DUMPBIN" >&5 $as_echo "$DUMPBIN" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi test -n "$DUMPBIN" && break done fi if test -z "$DUMPBIN"; then ac_ct_DUMPBIN=$DUMPBIN for ac_prog in dumpbin "link -dump" do # Extract the first word of "$ac_prog", so it can be a program name with args. set dummy $ac_prog; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_DUMPBIN+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_DUMPBIN"; then ac_cv_prog_ac_ct_DUMPBIN="$ac_ct_DUMPBIN" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_DUMPBIN="$ac_prog" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_DUMPBIN=$ac_cv_prog_ac_ct_DUMPBIN if test -n "$ac_ct_DUMPBIN"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_DUMPBIN" >&5 $as_echo "$ac_ct_DUMPBIN" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi test -n "$ac_ct_DUMPBIN" && break done if test "x$ac_ct_DUMPBIN" = x; then DUMPBIN=":" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac DUMPBIN=$ac_ct_DUMPBIN fi fi case `$DUMPBIN -symbols -headers /dev/null 2>&1 | sed '1q'` in *COFF*) DUMPBIN="$DUMPBIN -symbols -headers" ;; *) DUMPBIN=: ;; esac fi if test : != "$DUMPBIN"; then NM=$DUMPBIN fi fi test -z "$NM" && NM=nm { $as_echo "$as_me:${as_lineno-$LINENO}: checking the name lister ($NM) interface" >&5 $as_echo_n "checking the name lister ($NM) interface... " >&6; } if ${lt_cv_nm_interface+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_nm_interface="BSD nm" echo "int some_variable = 0;" > conftest.$ac_ext (eval echo "\"\$as_me:$LINENO: $ac_compile\"" >&5) (eval "$ac_compile" 2>conftest.err) cat conftest.err >&5 (eval echo "\"\$as_me:$LINENO: $NM \\\"conftest.$ac_objext\\\"\"" >&5) (eval "$NM \"conftest.$ac_objext\"" 2>conftest.err > conftest.out) cat conftest.err >&5 (eval echo "\"\$as_me:$LINENO: output\"" >&5) cat conftest.out >&5 if $GREP 'External.*some_variable' conftest.out > /dev/null; then lt_cv_nm_interface="MS dumpbin" fi rm -f conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_nm_interface" >&5 $as_echo "$lt_cv_nm_interface" >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether ln -s works" >&5 $as_echo_n "checking whether ln -s works... " >&6; } LN_S=$as_ln_s if test "$LN_S" = "ln -s"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no, using $LN_S" >&5 $as_echo "no, using $LN_S" >&6; } fi # find the maximum length of command line arguments { $as_echo "$as_me:${as_lineno-$LINENO}: checking the maximum length of command line arguments" >&5 $as_echo_n "checking the maximum length of command line arguments... " >&6; } if ${lt_cv_sys_max_cmd_len+:} false; then : $as_echo_n "(cached) " >&6 else i=0 teststring=ABCD case $build_os in msdosdjgpp*) # On DJGPP, this test can blow up pretty badly due to problems in libc # (any single argument exceeding 2000 bytes causes a buffer overrun # during glob expansion). Even if it were fixed, the result of this # check would be larger than it should be. lt_cv_sys_max_cmd_len=12288; # 12K is about right ;; gnu*) # Under GNU Hurd, this test is not required because there is # no limit to the length of command line arguments. # Libtool will interpret -1 as no limit whatsoever lt_cv_sys_max_cmd_len=-1; ;; cygwin* | mingw* | cegcc*) # On Win9x/ME, this test blows up -- it succeeds, but takes # about 5 minutes as the teststring grows exponentially. # Worse, since 9x/ME are not pre-emptively multitasking, # you end up with a "frozen" computer, even though with patience # the test eventually succeeds (with a max line length of 256k). # Instead, let's just punt: use the minimum linelength reported by # all of the supported platforms: 8192 (on NT/2K/XP). lt_cv_sys_max_cmd_len=8192; ;; mint*) # On MiNT this can take a long time and run out of memory. lt_cv_sys_max_cmd_len=8192; ;; amigaos*) # On AmigaOS with pdksh, this test takes hours, literally. # So we just punt and use a minimum line length of 8192. lt_cv_sys_max_cmd_len=8192; ;; bitrig* | darwin* | dragonfly* | freebsd* | netbsd* | openbsd*) # This has been around since 386BSD, at least. Likely further. if test -x /sbin/sysctl; then lt_cv_sys_max_cmd_len=`/sbin/sysctl -n kern.argmax` elif test -x /usr/sbin/sysctl; then lt_cv_sys_max_cmd_len=`/usr/sbin/sysctl -n kern.argmax` else lt_cv_sys_max_cmd_len=65536 # usable default for all BSDs fi # And add a safety zone lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \/ 4` lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \* 3` ;; interix*) # We know the value 262144 and hardcode it with a safety zone (like BSD) lt_cv_sys_max_cmd_len=196608 ;; os2*) # The test takes a long time on OS/2. lt_cv_sys_max_cmd_len=8192 ;; osf*) # Dr. Hans Ekkehard Plesser reports seeing a kernel panic running configure # due to this test when exec_disable_arg_limit is 1 on Tru64. It is not # nice to cause kernel panics so lets avoid the loop below. # First set a reasonable default. lt_cv_sys_max_cmd_len=16384 # if test -x /sbin/sysconfig; then case `/sbin/sysconfig -q proc exec_disable_arg_limit` in *1*) lt_cv_sys_max_cmd_len=-1 ;; esac fi ;; sco3.2v5*) lt_cv_sys_max_cmd_len=102400 ;; sysv5* | sco5v6* | sysv4.2uw2*) kargmax=`grep ARG_MAX /etc/conf/cf.d/stune 2>/dev/null` if test -n "$kargmax"; then lt_cv_sys_max_cmd_len=`echo $kargmax | sed 's/.*[ ]//'` else lt_cv_sys_max_cmd_len=32768 fi ;; *) lt_cv_sys_max_cmd_len=`(getconf ARG_MAX) 2> /dev/null` if test -n "$lt_cv_sys_max_cmd_len" && \ test undefined != "$lt_cv_sys_max_cmd_len"; then lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \/ 4` lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \* 3` else # Make teststring a little bigger before we do anything with it. # a 1K string should be a reasonable start. for i in 1 2 3 4 5 6 7 8; do teststring=$teststring$teststring done SHELL=${SHELL-${CONFIG_SHELL-/bin/sh}} # If test is not a shell built-in, we'll probably end up computing a # maximum length that is only half of the actual maximum length, but # we can't tell. while { test X`env echo "$teststring$teststring" 2>/dev/null` \ = "X$teststring$teststring"; } >/dev/null 2>&1 && test 17 != "$i" # 1/2 MB should be enough do i=`expr $i + 1` teststring=$teststring$teststring done # Only check the string length outside the loop. lt_cv_sys_max_cmd_len=`expr "X$teststring" : ".*" 2>&1` teststring= # Add a significant safety factor because C++ compilers can tack on # massive amounts of additional arguments before passing them to the # linker. It appears as though 1/2 is a usable value. lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \/ 2` fi ;; esac fi if test -n "$lt_cv_sys_max_cmd_len"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_sys_max_cmd_len" >&5 $as_echo "$lt_cv_sys_max_cmd_len" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: none" >&5 $as_echo "none" >&6; } fi max_cmd_len=$lt_cv_sys_max_cmd_len : ${CP="cp -f"} : ${MV="mv -f"} : ${RM="rm -f"} if ( (MAIL=60; unset MAIL) || exit) >/dev/null 2>&1; then lt_unset=unset else lt_unset=false fi # test EBCDIC or ASCII case `echo X|tr X '\101'` in A) # ASCII based system # \n is not interpreted correctly by Solaris 8 /usr/ucb/tr lt_SP2NL='tr \040 \012' lt_NL2SP='tr \015\012 \040\040' ;; *) # EBCDIC based system lt_SP2NL='tr \100 \n' lt_NL2SP='tr \r\n \100\100' ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: checking how to convert $build file names to $host format" >&5 $as_echo_n "checking how to convert $build file names to $host format... " >&6; } if ${lt_cv_to_host_file_cmd+:} false; then : $as_echo_n "(cached) " >&6 else case $host in *-*-mingw* ) case $build in *-*-mingw* ) # actually msys lt_cv_to_host_file_cmd=func_convert_file_msys_to_w32 ;; *-*-cygwin* ) lt_cv_to_host_file_cmd=func_convert_file_cygwin_to_w32 ;; * ) # otherwise, assume *nix lt_cv_to_host_file_cmd=func_convert_file_nix_to_w32 ;; esac ;; *-*-cygwin* ) case $build in *-*-mingw* ) # actually msys lt_cv_to_host_file_cmd=func_convert_file_msys_to_cygwin ;; *-*-cygwin* ) lt_cv_to_host_file_cmd=func_convert_file_noop ;; * ) # otherwise, assume *nix lt_cv_to_host_file_cmd=func_convert_file_nix_to_cygwin ;; esac ;; * ) # unhandled hosts (and "normal" native builds) lt_cv_to_host_file_cmd=func_convert_file_noop ;; esac fi to_host_file_cmd=$lt_cv_to_host_file_cmd { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_to_host_file_cmd" >&5 $as_echo "$lt_cv_to_host_file_cmd" >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: checking how to convert $build file names to toolchain format" >&5 $as_echo_n "checking how to convert $build file names to toolchain format... " >&6; } if ${lt_cv_to_tool_file_cmd+:} false; then : $as_echo_n "(cached) " >&6 else #assume ordinary cross tools, or native build. lt_cv_to_tool_file_cmd=func_convert_file_noop case $host in *-*-mingw* ) case $build in *-*-mingw* ) # actually msys lt_cv_to_tool_file_cmd=func_convert_file_msys_to_w32 ;; esac ;; esac fi to_tool_file_cmd=$lt_cv_to_tool_file_cmd { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_to_tool_file_cmd" >&5 $as_echo "$lt_cv_to_tool_file_cmd" >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $LD option to reload object files" >&5 $as_echo_n "checking for $LD option to reload object files... " >&6; } if ${lt_cv_ld_reload_flag+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_ld_reload_flag='-r' fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_ld_reload_flag" >&5 $as_echo "$lt_cv_ld_reload_flag" >&6; } reload_flag=$lt_cv_ld_reload_flag case $reload_flag in "" | " "*) ;; *) reload_flag=" $reload_flag" ;; esac reload_cmds='$LD$reload_flag -o $output$reload_objs' case $host_os in cygwin* | mingw* | pw32* | cegcc*) if test yes != "$GCC"; then reload_cmds=false fi ;; darwin*) if test yes = "$GCC"; then reload_cmds='$LTCC $LTCFLAGS -nostdlib $wl-r -o $output$reload_objs' else reload_cmds='$LD$reload_flag -o $output$reload_objs' fi ;; esac if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}objdump", so it can be a program name with args. set dummy ${ac_tool_prefix}objdump; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_OBJDUMP+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$OBJDUMP"; then ac_cv_prog_OBJDUMP="$OBJDUMP" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_OBJDUMP="${ac_tool_prefix}objdump" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi OBJDUMP=$ac_cv_prog_OBJDUMP if test -n "$OBJDUMP"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $OBJDUMP" >&5 $as_echo "$OBJDUMP" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_OBJDUMP"; then ac_ct_OBJDUMP=$OBJDUMP # Extract the first word of "objdump", so it can be a program name with args. set dummy objdump; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_OBJDUMP+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_OBJDUMP"; then ac_cv_prog_ac_ct_OBJDUMP="$ac_ct_OBJDUMP" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_OBJDUMP="objdump" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_OBJDUMP=$ac_cv_prog_ac_ct_OBJDUMP if test -n "$ac_ct_OBJDUMP"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_OBJDUMP" >&5 $as_echo "$ac_ct_OBJDUMP" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_OBJDUMP" = x; then OBJDUMP="false" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac OBJDUMP=$ac_ct_OBJDUMP fi else OBJDUMP="$ac_cv_prog_OBJDUMP" fi test -z "$OBJDUMP" && OBJDUMP=objdump { $as_echo "$as_me:${as_lineno-$LINENO}: checking how to recognize dependent libraries" >&5 $as_echo_n "checking how to recognize dependent libraries... " >&6; } if ${lt_cv_deplibs_check_method+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_file_magic_cmd='$MAGIC_CMD' lt_cv_file_magic_test_file= lt_cv_deplibs_check_method='unknown' # Need to set the preceding variable on all platforms that support # interlibrary dependencies. # 'none' -- dependencies not supported. # 'unknown' -- same as none, but documents that we really don't know. # 'pass_all' -- all dependencies passed with no checks. # 'test_compile' -- check by making test program. # 'file_magic [[regex]]' -- check by looking for files in library path # that responds to the $file_magic_cmd with a given extended regex. # If you have 'file' or equivalent on your system and you're not sure # whether 'pass_all' will *always* work, you probably want this one. case $host_os in aix[4-9]*) lt_cv_deplibs_check_method=pass_all ;; beos*) lt_cv_deplibs_check_method=pass_all ;; bsdi[45]*) lt_cv_deplibs_check_method='file_magic ELF [0-9][0-9]*-bit [ML]SB (shared object|dynamic lib)' lt_cv_file_magic_cmd='/usr/bin/file -L' lt_cv_file_magic_test_file=/shlib/libc.so ;; cygwin*) # func_win32_libid is a shell function defined in ltmain.sh lt_cv_deplibs_check_method='file_magic ^x86 archive import|^x86 DLL' lt_cv_file_magic_cmd='func_win32_libid' ;; mingw* | pw32*) # Base MSYS/MinGW do not provide the 'file' command needed by # func_win32_libid shell function, so use a weaker test based on 'objdump', # unless we find 'file', for example because we are cross-compiling. if ( file / ) >/dev/null 2>&1; then lt_cv_deplibs_check_method='file_magic ^x86 archive import|^x86 DLL' lt_cv_file_magic_cmd='func_win32_libid' else # Keep this pattern in sync with the one in func_win32_libid. lt_cv_deplibs_check_method='file_magic file format (pei*-i386(.*architecture: i386)?|pe-arm-wince|pe-x86-64)' lt_cv_file_magic_cmd='$OBJDUMP -f' fi ;; cegcc*) # use the weaker test based on 'objdump'. See mingw*. lt_cv_deplibs_check_method='file_magic file format pe-arm-.*little(.*architecture: arm)?' lt_cv_file_magic_cmd='$OBJDUMP -f' ;; darwin* | rhapsody*) lt_cv_deplibs_check_method=pass_all ;; freebsd* | dragonfly*) if echo __ELF__ | $CC -E - | $GREP __ELF__ > /dev/null; then case $host_cpu in i*86 ) # Not sure whether the presence of OpenBSD here was a mistake. # Let's accept both of them until this is cleared up. lt_cv_deplibs_check_method='file_magic (FreeBSD|OpenBSD|DragonFly)/i[3-9]86 (compact )?demand paged shared library' lt_cv_file_magic_cmd=/usr/bin/file lt_cv_file_magic_test_file=`echo /usr/lib/libc.so.*` ;; esac else lt_cv_deplibs_check_method=pass_all fi ;; haiku*) lt_cv_deplibs_check_method=pass_all ;; hpux10.20* | hpux11*) lt_cv_file_magic_cmd=/usr/bin/file case $host_cpu in ia64*) lt_cv_deplibs_check_method='file_magic (s[0-9][0-9][0-9]|ELF-[0-9][0-9]) shared object file - IA64' lt_cv_file_magic_test_file=/usr/lib/hpux32/libc.so ;; hppa*64*) lt_cv_deplibs_check_method='file_magic (s[0-9][0-9][0-9]|ELF[ -][0-9][0-9])(-bit)?( [LM]SB)? shared object( file)?[, -]* PA-RISC [0-9]\.[0-9]' lt_cv_file_magic_test_file=/usr/lib/pa20_64/libc.sl ;; *) lt_cv_deplibs_check_method='file_magic (s[0-9][0-9][0-9]|PA-RISC[0-9]\.[0-9]) shared library' lt_cv_file_magic_test_file=/usr/lib/libc.sl ;; esac ;; interix[3-9]*) # PIC code is broken on Interix 3.x, that's why |\.a not |_pic\.a here lt_cv_deplibs_check_method='match_pattern /lib[^/]+(\.so|\.a)$' ;; irix5* | irix6* | nonstopux*) case $LD in *-32|*"-32 ") libmagic=32-bit;; *-n32|*"-n32 ") libmagic=N32;; *-64|*"-64 ") libmagic=64-bit;; *) libmagic=never-match;; esac lt_cv_deplibs_check_method=pass_all ;; # This must be glibc/ELF. linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*) lt_cv_deplibs_check_method=pass_all ;; netbsd* | netbsdelf*-gnu) if echo __ELF__ | $CC -E - | $GREP __ELF__ > /dev/null; then lt_cv_deplibs_check_method='match_pattern /lib[^/]+(\.so\.[0-9]+\.[0-9]+|_pic\.a)$' else lt_cv_deplibs_check_method='match_pattern /lib[^/]+(\.so|_pic\.a)$' fi ;; newos6*) lt_cv_deplibs_check_method='file_magic ELF [0-9][0-9]*-bit [ML]SB (executable|dynamic lib)' lt_cv_file_magic_cmd=/usr/bin/file lt_cv_file_magic_test_file=/usr/lib/libnls.so ;; *nto* | *qnx*) lt_cv_deplibs_check_method=pass_all ;; openbsd* | bitrig*) if test -z "`echo __ELF__ | $CC -E - | $GREP __ELF__`"; then lt_cv_deplibs_check_method='match_pattern /lib[^/]+(\.so\.[0-9]+\.[0-9]+|\.so|_pic\.a)$' else lt_cv_deplibs_check_method='match_pattern /lib[^/]+(\.so\.[0-9]+\.[0-9]+|_pic\.a)$' fi ;; osf3* | osf4* | osf5*) lt_cv_deplibs_check_method=pass_all ;; rdos*) lt_cv_deplibs_check_method=pass_all ;; solaris*) lt_cv_deplibs_check_method=pass_all ;; sysv5* | sco3.2v5* | sco5v6* | unixware* | OpenUNIX* | sysv4*uw2*) lt_cv_deplibs_check_method=pass_all ;; sysv4 | sysv4.3*) case $host_vendor in motorola) lt_cv_deplibs_check_method='file_magic ELF [0-9][0-9]*-bit [ML]SB (shared object|dynamic lib) M[0-9][0-9]* Version [0-9]' lt_cv_file_magic_test_file=`echo /usr/lib/libc.so*` ;; ncr) lt_cv_deplibs_check_method=pass_all ;; sequent) lt_cv_file_magic_cmd='/bin/file' lt_cv_deplibs_check_method='file_magic ELF [0-9][0-9]*-bit [LM]SB (shared object|dynamic lib )' ;; sni) lt_cv_file_magic_cmd='/bin/file' lt_cv_deplibs_check_method="file_magic ELF [0-9][0-9]*-bit [LM]SB dynamic lib" lt_cv_file_magic_test_file=/lib/libc.so ;; siemens) lt_cv_deplibs_check_method=pass_all ;; pc) lt_cv_deplibs_check_method=pass_all ;; esac ;; tpf*) lt_cv_deplibs_check_method=pass_all ;; os2*) lt_cv_deplibs_check_method=pass_all ;; esac fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_deplibs_check_method" >&5 $as_echo "$lt_cv_deplibs_check_method" >&6; } file_magic_glob= want_nocaseglob=no if test "$build" = "$host"; then case $host_os in mingw* | pw32*) if ( shopt | grep nocaseglob ) >/dev/null 2>&1; then want_nocaseglob=yes else file_magic_glob=`echo aAbBcCdDeEfFgGhHiIjJkKlLmMnNoOpPqQrRsStTuUvVwWxXyYzZ | $SED -e "s/\(..\)/s\/[\1]\/[\1]\/g;/g"` fi ;; esac fi file_magic_cmd=$lt_cv_file_magic_cmd deplibs_check_method=$lt_cv_deplibs_check_method test -z "$deplibs_check_method" && deplibs_check_method=unknown if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}dlltool", so it can be a program name with args. set dummy ${ac_tool_prefix}dlltool; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_DLLTOOL+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$DLLTOOL"; then ac_cv_prog_DLLTOOL="$DLLTOOL" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_DLLTOOL="${ac_tool_prefix}dlltool" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi DLLTOOL=$ac_cv_prog_DLLTOOL if test -n "$DLLTOOL"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $DLLTOOL" >&5 $as_echo "$DLLTOOL" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_DLLTOOL"; then ac_ct_DLLTOOL=$DLLTOOL # Extract the first word of "dlltool", so it can be a program name with args. set dummy dlltool; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_DLLTOOL+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_DLLTOOL"; then ac_cv_prog_ac_ct_DLLTOOL="$ac_ct_DLLTOOL" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_DLLTOOL="dlltool" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_DLLTOOL=$ac_cv_prog_ac_ct_DLLTOOL if test -n "$ac_ct_DLLTOOL"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_DLLTOOL" >&5 $as_echo "$ac_ct_DLLTOOL" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_DLLTOOL" = x; then DLLTOOL="false" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac DLLTOOL=$ac_ct_DLLTOOL fi else DLLTOOL="$ac_cv_prog_DLLTOOL" fi test -z "$DLLTOOL" && DLLTOOL=dlltool { $as_echo "$as_me:${as_lineno-$LINENO}: checking how to associate runtime and link libraries" >&5 $as_echo_n "checking how to associate runtime and link libraries... " >&6; } if ${lt_cv_sharedlib_from_linklib_cmd+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_sharedlib_from_linklib_cmd='unknown' case $host_os in cygwin* | mingw* | pw32* | cegcc*) # two different shell functions defined in ltmain.sh; # decide which one to use based on capabilities of $DLLTOOL case `$DLLTOOL --help 2>&1` in *--identify-strict*) lt_cv_sharedlib_from_linklib_cmd=func_cygming_dll_for_implib ;; *) lt_cv_sharedlib_from_linklib_cmd=func_cygming_dll_for_implib_fallback ;; esac ;; *) # fallback: assume linklib IS sharedlib lt_cv_sharedlib_from_linklib_cmd=$ECHO ;; esac fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_sharedlib_from_linklib_cmd" >&5 $as_echo "$lt_cv_sharedlib_from_linklib_cmd" >&6; } sharedlib_from_linklib_cmd=$lt_cv_sharedlib_from_linklib_cmd test -z "$sharedlib_from_linklib_cmd" && sharedlib_from_linklib_cmd=$ECHO if test -n "$ac_tool_prefix"; then for ac_prog in ar do # Extract the first word of "$ac_tool_prefix$ac_prog", so it can be a program name with args. set dummy $ac_tool_prefix$ac_prog; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_AR+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$AR"; then ac_cv_prog_AR="$AR" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_AR="$ac_tool_prefix$ac_prog" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi AR=$ac_cv_prog_AR if test -n "$AR"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $AR" >&5 $as_echo "$AR" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi test -n "$AR" && break done fi if test -z "$AR"; then ac_ct_AR=$AR for ac_prog in ar do # Extract the first word of "$ac_prog", so it can be a program name with args. set dummy $ac_prog; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_AR+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_AR"; then ac_cv_prog_ac_ct_AR="$ac_ct_AR" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_AR="$ac_prog" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_AR=$ac_cv_prog_ac_ct_AR if test -n "$ac_ct_AR"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_AR" >&5 $as_echo "$ac_ct_AR" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi test -n "$ac_ct_AR" && break done if test "x$ac_ct_AR" = x; then AR="false" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac AR=$ac_ct_AR fi fi : ${AR=ar} : ${AR_FLAGS=cru} { $as_echo "$as_me:${as_lineno-$LINENO}: checking for archiver @FILE support" >&5 $as_echo_n "checking for archiver @FILE support... " >&6; } if ${lt_cv_ar_at_file+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_ar_at_file=no cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_cxx_try_compile "$LINENO"; then : echo conftest.$ac_objext > conftest.lst lt_ar_try='$AR $AR_FLAGS libconftest.a @conftest.lst >&5' { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$lt_ar_try\""; } >&5 (eval $lt_ar_try) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } if test 0 -eq "$ac_status"; then # Ensure the archiver fails upon bogus file names. rm -f conftest.$ac_objext libconftest.a { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$lt_ar_try\""; } >&5 (eval $lt_ar_try) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } if test 0 -ne "$ac_status"; then lt_cv_ar_at_file=@ fi fi rm -f conftest.* libconftest.a fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_ar_at_file" >&5 $as_echo "$lt_cv_ar_at_file" >&6; } if test no = "$lt_cv_ar_at_file"; then archiver_list_spec= else archiver_list_spec=$lt_cv_ar_at_file fi if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}strip", so it can be a program name with args. set dummy ${ac_tool_prefix}strip; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_STRIP+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$STRIP"; then ac_cv_prog_STRIP="$STRIP" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_STRIP="${ac_tool_prefix}strip" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi STRIP=$ac_cv_prog_STRIP if test -n "$STRIP"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $STRIP" >&5 $as_echo "$STRIP" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_STRIP"; then ac_ct_STRIP=$STRIP # Extract the first word of "strip", so it can be a program name with args. set dummy strip; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_STRIP+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_STRIP"; then ac_cv_prog_ac_ct_STRIP="$ac_ct_STRIP" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_STRIP="strip" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_STRIP=$ac_cv_prog_ac_ct_STRIP if test -n "$ac_ct_STRIP"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_STRIP" >&5 $as_echo "$ac_ct_STRIP" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_STRIP" = x; then STRIP=":" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac STRIP=$ac_ct_STRIP fi else STRIP="$ac_cv_prog_STRIP" fi test -z "$STRIP" && STRIP=: if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}ranlib", so it can be a program name with args. set dummy ${ac_tool_prefix}ranlib; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_RANLIB+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$RANLIB"; then ac_cv_prog_RANLIB="$RANLIB" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_RANLIB="${ac_tool_prefix}ranlib" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi RANLIB=$ac_cv_prog_RANLIB if test -n "$RANLIB"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $RANLIB" >&5 $as_echo "$RANLIB" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_RANLIB"; then ac_ct_RANLIB=$RANLIB # Extract the first word of "ranlib", so it can be a program name with args. set dummy ranlib; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_RANLIB+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_RANLIB"; then ac_cv_prog_ac_ct_RANLIB="$ac_ct_RANLIB" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_RANLIB="ranlib" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_RANLIB=$ac_cv_prog_ac_ct_RANLIB if test -n "$ac_ct_RANLIB"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_RANLIB" >&5 $as_echo "$ac_ct_RANLIB" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_RANLIB" = x; then RANLIB=":" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac RANLIB=$ac_ct_RANLIB fi else RANLIB="$ac_cv_prog_RANLIB" fi test -z "$RANLIB" && RANLIB=: # Determine commands to create old-style static archives. old_archive_cmds='$AR $AR_FLAGS $oldlib$oldobjs' old_postinstall_cmds='chmod 644 $oldlib' old_postuninstall_cmds= if test -n "$RANLIB"; then case $host_os in bitrig* | openbsd*) old_postinstall_cmds="$old_postinstall_cmds~\$RANLIB -t \$tool_oldlib" ;; *) old_postinstall_cmds="$old_postinstall_cmds~\$RANLIB \$tool_oldlib" ;; esac old_archive_cmds="$old_archive_cmds~\$RANLIB \$tool_oldlib" fi case $host_os in darwin*) lock_old_archive_extraction=yes ;; *) lock_old_archive_extraction=no ;; esac # If no C compiler was specified, use CC. LTCC=${LTCC-"$CC"} # If no C compiler flags were specified, use CFLAGS. LTCFLAGS=${LTCFLAGS-"$CFLAGS"} # Allow CC to be a program name with arguments. compiler=$CC # Check for command to grab the raw symbol name followed by C symbol from nm. { $as_echo "$as_me:${as_lineno-$LINENO}: checking command to parse $NM output from $compiler object" >&5 $as_echo_n "checking command to parse $NM output from $compiler object... " >&6; } if ${lt_cv_sys_global_symbol_pipe+:} false; then : $as_echo_n "(cached) " >&6 else # These are sane defaults that work on at least a few old systems. # [They come from Ultrix. What could be older than Ultrix?!! ;)] # Character class describing NM global symbol codes. symcode='[BCDEGRST]' # Regexp to match symbols that can be accessed directly from C. sympat='\([_A-Za-z][_A-Za-z0-9]*\)' # Define system-specific variables. case $host_os in aix*) symcode='[BCDT]' ;; cygwin* | mingw* | pw32* | cegcc*) symcode='[ABCDGISTW]' ;; hpux*) if test ia64 = "$host_cpu"; then symcode='[ABCDEGRST]' fi ;; irix* | nonstopux*) symcode='[BCDEGRST]' ;; osf*) symcode='[BCDEGQRST]' ;; solaris*) symcode='[BDRT]' ;; sco3.2v5*) symcode='[DT]' ;; sysv4.2uw2*) symcode='[DT]' ;; sysv5* | sco5v6* | unixware* | OpenUNIX*) symcode='[ABDT]' ;; sysv4) symcode='[DFNSTU]' ;; esac # If we're using GNU nm, then use its standard symbol codes. case `$NM -V 2>&1` in *GNU* | *'with BFD'*) symcode='[ABCDGIRSTW]' ;; esac if test "$lt_cv_nm_interface" = "MS dumpbin"; then # Gets list of data symbols to import. lt_cv_sys_global_symbol_to_import="sed -n -e 's/^I .* \(.*\)$/\1/p'" # Adjust the below global symbol transforms to fixup imported variables. lt_cdecl_hook=" -e 's/^I .* \(.*\)$/extern __declspec(dllimport) char \1;/p'" lt_c_name_hook=" -e 's/^I .* \(.*\)$/ {\"\1\", (void *) 0},/p'" lt_c_name_lib_hook="\ -e 's/^I .* \(lib.*\)$/ {\"\1\", (void *) 0},/p'\ -e 's/^I .* \(.*\)$/ {\"lib\1\", (void *) 0},/p'" else # Disable hooks by default. lt_cv_sys_global_symbol_to_import= lt_cdecl_hook= lt_c_name_hook= lt_c_name_lib_hook= fi # Transform an extracted symbol line into a proper C declaration. # Some systems (esp. on ia64) link data and code symbols differently, # so use this general approach. lt_cv_sys_global_symbol_to_cdecl="sed -n"\ $lt_cdecl_hook\ " -e 's/^T .* \(.*\)$/extern int \1();/p'"\ " -e 's/^$symcode$symcode* .* \(.*\)$/extern char \1;/p'" # Transform an extracted symbol line into symbol name and symbol address lt_cv_sys_global_symbol_to_c_name_address="sed -n"\ $lt_c_name_hook\ " -e 's/^: \(.*\) .*$/ {\"\1\", (void *) 0},/p'"\ " -e 's/^$symcode$symcode* .* \(.*\)$/ {\"\1\", (void *) \&\1},/p'" # Transform an extracted symbol line into symbol name with lib prefix and # symbol address. lt_cv_sys_global_symbol_to_c_name_address_lib_prefix="sed -n"\ $lt_c_name_lib_hook\ " -e 's/^: \(.*\) .*$/ {\"\1\", (void *) 0},/p'"\ " -e 's/^$symcode$symcode* .* \(lib.*\)$/ {\"\1\", (void *) \&\1},/p'"\ " -e 's/^$symcode$symcode* .* \(.*\)$/ {\"lib\1\", (void *) \&\1},/p'" # Handle CRLF in mingw tool chain opt_cr= case $build_os in mingw*) opt_cr=`$ECHO 'x\{0,1\}' | tr x '\015'` # option cr in regexp ;; esac # Try without a prefix underscore, then with it. for ac_symprfx in "" "_"; do # Transform symcode, sympat, and symprfx into a raw symbol and a C symbol. symxfrm="\\1 $ac_symprfx\\2 \\2" # Write the raw and C identifiers. if test "$lt_cv_nm_interface" = "MS dumpbin"; then # Fake it for dumpbin and say T for any non-static function, # D for any global variable and I for any imported variable. # Also find C++ and __fastcall symbols from MSVC++, # which start with @ or ?. lt_cv_sys_global_symbol_pipe="$AWK '"\ " {last_section=section; section=\$ 3};"\ " /^COFF SYMBOL TABLE/{for(i in hide) delete hide[i]};"\ " /Section length .*#relocs.*(pick any)/{hide[last_section]=1};"\ " /^ *Symbol name *: /{split(\$ 0,sn,\":\"); si=substr(sn[2],2)};"\ " /^ *Type *: code/{print \"T\",si,substr(si,length(prfx))};"\ " /^ *Type *: data/{print \"I\",si,substr(si,length(prfx))};"\ " \$ 0!~/External *\|/{next};"\ " / 0+ UNDEF /{next}; / UNDEF \([^|]\)*()/{next};"\ " {if(hide[section]) next};"\ " {f=\"D\"}; \$ 0~/\(\).*\|/{f=\"T\"};"\ " {split(\$ 0,a,/\||\r/); split(a[2],s)};"\ " s[1]~/^[@?]/{print f,s[1],s[1]; next};"\ " s[1]~prfx {split(s[1],t,\"@\"); print f,t[1],substr(t[1],length(prfx))}"\ " ' prfx=^$ac_symprfx" else lt_cv_sys_global_symbol_pipe="sed -n -e 's/^.*[ ]\($symcode$symcode*\)[ ][ ]*$ac_symprfx$sympat$opt_cr$/$symxfrm/p'" fi lt_cv_sys_global_symbol_pipe="$lt_cv_sys_global_symbol_pipe | sed '/ __gnu_lto/d'" # Check to see that the pipe works correctly. pipe_works=no rm -f conftest* cat > conftest.$ac_ext <<_LT_EOF #ifdef __cplusplus extern "C" { #endif char nm_test_var; void nm_test_func(void); void nm_test_func(void){} #ifdef __cplusplus } #endif int main(){nm_test_var='a';nm_test_func();return(0);} _LT_EOF if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 (eval $ac_compile) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then # Now try to grab the symbols. nlist=conftest.nm if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$NM conftest.$ac_objext \| "$lt_cv_sys_global_symbol_pipe" \> $nlist\""; } >&5 (eval $NM conftest.$ac_objext \| "$lt_cv_sys_global_symbol_pipe" \> $nlist) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } && test -s "$nlist"; then # Try sorting and uniquifying the output. if sort "$nlist" | uniq > "$nlist"T; then mv -f "$nlist"T "$nlist" else rm -f "$nlist"T fi # Make sure that we snagged all the symbols we need. if $GREP ' nm_test_var$' "$nlist" >/dev/null; then if $GREP ' nm_test_func$' "$nlist" >/dev/null; then cat <<_LT_EOF > conftest.$ac_ext /* Keep this code in sync between libtool.m4, ltmain, lt_system.h, and tests. */ #if defined _WIN32 || defined __CYGWIN__ || defined _WIN32_WCE /* DATA imports from DLLs on WIN32 can't be const, because runtime relocations are performed -- see ld's documentation on pseudo-relocs. */ # define LT_DLSYM_CONST #elif defined __osf__ /* This system does not cope well with relocations in const data. */ # define LT_DLSYM_CONST #else # define LT_DLSYM_CONST const #endif #ifdef __cplusplus extern "C" { #endif _LT_EOF # Now generate the symbol file. eval "$lt_cv_sys_global_symbol_to_cdecl"' < "$nlist" | $GREP -v main >> conftest.$ac_ext' cat <<_LT_EOF >> conftest.$ac_ext /* The mapping between symbol names and symbols. */ LT_DLSYM_CONST struct { const char *name; void *address; } lt__PROGRAM__LTX_preloaded_symbols[] = { { "@PROGRAM@", (void *) 0 }, _LT_EOF $SED "s/^$symcode$symcode* .* \(.*\)$/ {\"\1\", (void *) \&\1},/" < "$nlist" | $GREP -v main >> conftest.$ac_ext cat <<\_LT_EOF >> conftest.$ac_ext {0, (void *) 0} }; /* This works around a problem in FreeBSD linker */ #ifdef FREEBSD_WORKAROUND static const void *lt_preloaded_setup() { return lt__PROGRAM__LTX_preloaded_symbols; } #endif #ifdef __cplusplus } #endif _LT_EOF # Now try linking the two files. mv conftest.$ac_objext conftstm.$ac_objext lt_globsym_save_LIBS=$LIBS lt_globsym_save_CFLAGS=$CFLAGS LIBS=conftstm.$ac_objext CFLAGS="$CFLAGS$lt_prog_compiler_no_builtin_flag" if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_link\""; } >&5 (eval $ac_link) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } && test -s conftest$ac_exeext; then pipe_works=yes fi LIBS=$lt_globsym_save_LIBS CFLAGS=$lt_globsym_save_CFLAGS else echo "cannot find nm_test_func in $nlist" >&5 fi else echo "cannot find nm_test_var in $nlist" >&5 fi else echo "cannot run $lt_cv_sys_global_symbol_pipe" >&5 fi else echo "$progname: failed program was:" >&5 cat conftest.$ac_ext >&5 fi rm -rf conftest* conftst* # Do not use the global_symbol_pipe unless it works. if test yes = "$pipe_works"; then break else lt_cv_sys_global_symbol_pipe= fi done fi if test -z "$lt_cv_sys_global_symbol_pipe"; then lt_cv_sys_global_symbol_to_cdecl= fi if test -z "$lt_cv_sys_global_symbol_pipe$lt_cv_sys_global_symbol_to_cdecl"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: failed" >&5 $as_echo "failed" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: ok" >&5 $as_echo "ok" >&6; } fi # Response file support. if test "$lt_cv_nm_interface" = "MS dumpbin"; then nm_file_list_spec='@' elif $NM --help 2>/dev/null | grep '[@]FILE' >/dev/null; then nm_file_list_spec='@' fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for sysroot" >&5 $as_echo_n "checking for sysroot... " >&6; } # Check whether --with-sysroot was given. if test "${with_sysroot+set}" = set; then : withval=$with_sysroot; else with_sysroot=no fi lt_sysroot= case $with_sysroot in #( yes) if test yes = "$GCC"; then lt_sysroot=`$CC --print-sysroot 2>/dev/null` fi ;; #( /*) lt_sysroot=`echo "$with_sysroot" | sed -e "$sed_quote_subst"` ;; #( no|'') ;; #( *) { $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_sysroot" >&5 $as_echo "$with_sysroot" >&6; } as_fn_error $? "The sysroot must be an absolute path." "$LINENO" 5 ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: result: ${lt_sysroot:-no}" >&5 $as_echo "${lt_sysroot:-no}" >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: checking for a working dd" >&5 $as_echo_n "checking for a working dd... " >&6; } if ${ac_cv_path_lt_DD+:} false; then : $as_echo_n "(cached) " >&6 else printf 0123456789abcdef0123456789abcdef >conftest.i cat conftest.i conftest.i >conftest2.i : ${lt_DD:=$DD} if test -z "$lt_DD"; then ac_path_lt_DD_found=false # Loop through the user's path and test for each of PROGNAME-LIST as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_prog in dd; do for ac_exec_ext in '' $ac_executable_extensions; do ac_path_lt_DD="$as_dir/$ac_prog$ac_exec_ext" as_fn_executable_p "$ac_path_lt_DD" || continue if "$ac_path_lt_DD" bs=32 count=1 conftest.out 2>/dev/null; then cmp -s conftest.i conftest.out \ && ac_cv_path_lt_DD="$ac_path_lt_DD" ac_path_lt_DD_found=: fi $ac_path_lt_DD_found && break 3 done done done IFS=$as_save_IFS if test -z "$ac_cv_path_lt_DD"; then : fi else ac_cv_path_lt_DD=$lt_DD fi rm -f conftest.i conftest2.i conftest.out fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_path_lt_DD" >&5 $as_echo "$ac_cv_path_lt_DD" >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: checking how to truncate binary pipes" >&5 $as_echo_n "checking how to truncate binary pipes... " >&6; } if ${lt_cv_truncate_bin+:} false; then : $as_echo_n "(cached) " >&6 else printf 0123456789abcdef0123456789abcdef >conftest.i cat conftest.i conftest.i >conftest2.i lt_cv_truncate_bin= if "$ac_cv_path_lt_DD" bs=32 count=1 conftest.out 2>/dev/null; then cmp -s conftest.i conftest.out \ && lt_cv_truncate_bin="$ac_cv_path_lt_DD bs=4096 count=1" fi rm -f conftest.i conftest2.i conftest.out test -z "$lt_cv_truncate_bin" && lt_cv_truncate_bin="$SED -e 4q" fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_truncate_bin" >&5 $as_echo "$lt_cv_truncate_bin" >&6; } # Calculate cc_basename. Skip known compiler wrappers and cross-prefix. func_cc_basename () { for cc_temp in $*""; do case $cc_temp in compile | *[\\/]compile | ccache | *[\\/]ccache ) ;; distcc | *[\\/]distcc | purify | *[\\/]purify ) ;; \-*) ;; *) break;; esac done func_cc_basename_result=`$ECHO "$cc_temp" | $SED "s%.*/%%; s%^$host_alias-%%"` } # Check whether --enable-libtool-lock was given. if test "${enable_libtool_lock+set}" = set; then : enableval=$enable_libtool_lock; fi test no = "$enable_libtool_lock" || enable_libtool_lock=yes # Some flags need to be propagated to the compiler or linker for good # libtool support. case $host in ia64-*-hpux*) # Find out what ABI is being produced by ac_compile, and set mode # options accordingly. echo 'int i;' > conftest.$ac_ext if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 (eval $ac_compile) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then case `/usr/bin/file conftest.$ac_objext` in *ELF-32*) HPUX_IA64_MODE=32 ;; *ELF-64*) HPUX_IA64_MODE=64 ;; esac fi rm -rf conftest* ;; *-*-irix6*) # Find out what ABI is being produced by ac_compile, and set linker # options accordingly. echo '#line '$LINENO' "configure"' > conftest.$ac_ext if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 (eval $ac_compile) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then if test yes = "$lt_cv_prog_gnu_ld"; then case `/usr/bin/file conftest.$ac_objext` in *32-bit*) LD="${LD-ld} -melf32bsmip" ;; *N32*) LD="${LD-ld} -melf32bmipn32" ;; *64-bit*) LD="${LD-ld} -melf64bmip" ;; esac else case `/usr/bin/file conftest.$ac_objext` in *32-bit*) LD="${LD-ld} -32" ;; *N32*) LD="${LD-ld} -n32" ;; *64-bit*) LD="${LD-ld} -64" ;; esac fi fi rm -rf conftest* ;; mips64*-*linux*) # Find out what ABI is being produced by ac_compile, and set linker # options accordingly. echo '#line '$LINENO' "configure"' > conftest.$ac_ext if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 (eval $ac_compile) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then emul=elf case `/usr/bin/file conftest.$ac_objext` in *32-bit*) emul="${emul}32" ;; *64-bit*) emul="${emul}64" ;; esac case `/usr/bin/file conftest.$ac_objext` in *MSB*) emul="${emul}btsmip" ;; *LSB*) emul="${emul}ltsmip" ;; esac case `/usr/bin/file conftest.$ac_objext` in *N32*) emul="${emul}n32" ;; esac LD="${LD-ld} -m $emul" fi rm -rf conftest* ;; x86_64-*kfreebsd*-gnu|x86_64-*linux*|powerpc*-*linux*| \ s390*-*linux*|s390*-*tpf*|sparc*-*linux*) # Find out what ABI is being produced by ac_compile, and set linker # options accordingly. Note that the listed cases only cover the # situations where additional linker options are needed (such as when # doing 32-bit compilation for a host where ld defaults to 64-bit, or # vice versa); the common cases where no linker options are needed do # not appear in the list. echo 'int i;' > conftest.$ac_ext if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 (eval $ac_compile) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then case `/usr/bin/file conftest.o` in *32-bit*) case $host in x86_64-*kfreebsd*-gnu) LD="${LD-ld} -m elf_i386_fbsd" ;; x86_64-*linux*) case `/usr/bin/file conftest.o` in *x86-64*) LD="${LD-ld} -m elf32_x86_64" ;; *) LD="${LD-ld} -m elf_i386" ;; esac ;; powerpc64le-*linux*) LD="${LD-ld} -m elf32lppclinux" ;; powerpc64-*linux*) LD="${LD-ld} -m elf32ppclinux" ;; s390x-*linux*) LD="${LD-ld} -m elf_s390" ;; sparc64-*linux*) LD="${LD-ld} -m elf32_sparc" ;; esac ;; *64-bit*) case $host in x86_64-*kfreebsd*-gnu) LD="${LD-ld} -m elf_x86_64_fbsd" ;; x86_64-*linux*) LD="${LD-ld} -m elf_x86_64" ;; powerpcle-*linux*) LD="${LD-ld} -m elf64lppc" ;; powerpc-*linux*) LD="${LD-ld} -m elf64ppc" ;; s390*-*linux*|s390*-*tpf*) LD="${LD-ld} -m elf64_s390" ;; sparc*-*linux*) LD="${LD-ld} -m elf64_sparc" ;; esac ;; esac fi rm -rf conftest* ;; *-*-sco3.2v5*) # On SCO OpenServer 5, we need -belf to get full-featured binaries. SAVE_CFLAGS=$CFLAGS CFLAGS="$CFLAGS -belf" { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether the C compiler needs -belf" >&5 $as_echo_n "checking whether the C compiler needs -belf... " >&6; } if ${lt_cv_cc_needs_belf+:} false; then : $as_echo_n "(cached) " >&6 else ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : lt_cv_cc_needs_belf=yes else lt_cv_cc_needs_belf=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_cc_needs_belf" >&5 $as_echo "$lt_cv_cc_needs_belf" >&6; } if test yes != "$lt_cv_cc_needs_belf"; then # this is probably gcc 2.8.0, egcs 1.0 or newer; no need for -belf CFLAGS=$SAVE_CFLAGS fi ;; *-*solaris*) # Find out what ABI is being produced by ac_compile, and set linker # options accordingly. echo 'int i;' > conftest.$ac_ext if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 (eval $ac_compile) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then case `/usr/bin/file conftest.o` in *64-bit*) case $lt_cv_prog_gnu_ld in yes*) case $host in i?86-*-solaris*|x86_64-*-solaris*) LD="${LD-ld} -m elf_x86_64" ;; sparc*-*-solaris*) LD="${LD-ld} -m elf64_sparc" ;; esac # GNU ld 2.21 introduced _sol2 emulations. Use them if available. if ${LD-ld} -V | grep _sol2 >/dev/null 2>&1; then LD=${LD-ld}_sol2 fi ;; *) if ${LD-ld} -64 -r -o conftest2.o conftest.o >/dev/null 2>&1; then LD="${LD-ld} -64" fi ;; esac ;; esac fi rm -rf conftest* ;; esac need_locks=$enable_libtool_lock if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}mt", so it can be a program name with args. set dummy ${ac_tool_prefix}mt; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_MANIFEST_TOOL+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$MANIFEST_TOOL"; then ac_cv_prog_MANIFEST_TOOL="$MANIFEST_TOOL" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_MANIFEST_TOOL="${ac_tool_prefix}mt" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi MANIFEST_TOOL=$ac_cv_prog_MANIFEST_TOOL if test -n "$MANIFEST_TOOL"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $MANIFEST_TOOL" >&5 $as_echo "$MANIFEST_TOOL" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_MANIFEST_TOOL"; then ac_ct_MANIFEST_TOOL=$MANIFEST_TOOL # Extract the first word of "mt", so it can be a program name with args. set dummy mt; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_MANIFEST_TOOL+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_MANIFEST_TOOL"; then ac_cv_prog_ac_ct_MANIFEST_TOOL="$ac_ct_MANIFEST_TOOL" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_MANIFEST_TOOL="mt" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_MANIFEST_TOOL=$ac_cv_prog_ac_ct_MANIFEST_TOOL if test -n "$ac_ct_MANIFEST_TOOL"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_MANIFEST_TOOL" >&5 $as_echo "$ac_ct_MANIFEST_TOOL" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_MANIFEST_TOOL" = x; then MANIFEST_TOOL=":" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac MANIFEST_TOOL=$ac_ct_MANIFEST_TOOL fi else MANIFEST_TOOL="$ac_cv_prog_MANIFEST_TOOL" fi test -z "$MANIFEST_TOOL" && MANIFEST_TOOL=mt { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $MANIFEST_TOOL is a manifest tool" >&5 $as_echo_n "checking if $MANIFEST_TOOL is a manifest tool... " >&6; } if ${lt_cv_path_mainfest_tool+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_path_mainfest_tool=no echo "$as_me:$LINENO: $MANIFEST_TOOL '-?'" >&5 $MANIFEST_TOOL '-?' 2>conftest.err > conftest.out cat conftest.err >&5 if $GREP 'Manifest Tool' conftest.out > /dev/null; then lt_cv_path_mainfest_tool=yes fi rm -f conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_path_mainfest_tool" >&5 $as_echo "$lt_cv_path_mainfest_tool" >&6; } if test yes != "$lt_cv_path_mainfest_tool"; then MANIFEST_TOOL=: fi case $host_os in rhapsody* | darwin*) if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}dsymutil", so it can be a program name with args. set dummy ${ac_tool_prefix}dsymutil; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_DSYMUTIL+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$DSYMUTIL"; then ac_cv_prog_DSYMUTIL="$DSYMUTIL" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_DSYMUTIL="${ac_tool_prefix}dsymutil" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi DSYMUTIL=$ac_cv_prog_DSYMUTIL if test -n "$DSYMUTIL"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $DSYMUTIL" >&5 $as_echo "$DSYMUTIL" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_DSYMUTIL"; then ac_ct_DSYMUTIL=$DSYMUTIL # Extract the first word of "dsymutil", so it can be a program name with args. set dummy dsymutil; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_DSYMUTIL+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_DSYMUTIL"; then ac_cv_prog_ac_ct_DSYMUTIL="$ac_ct_DSYMUTIL" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_DSYMUTIL="dsymutil" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_DSYMUTIL=$ac_cv_prog_ac_ct_DSYMUTIL if test -n "$ac_ct_DSYMUTIL"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_DSYMUTIL" >&5 $as_echo "$ac_ct_DSYMUTIL" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_DSYMUTIL" = x; then DSYMUTIL=":" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac DSYMUTIL=$ac_ct_DSYMUTIL fi else DSYMUTIL="$ac_cv_prog_DSYMUTIL" fi if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}nmedit", so it can be a program name with args. set dummy ${ac_tool_prefix}nmedit; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_NMEDIT+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$NMEDIT"; then ac_cv_prog_NMEDIT="$NMEDIT" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_NMEDIT="${ac_tool_prefix}nmedit" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi NMEDIT=$ac_cv_prog_NMEDIT if test -n "$NMEDIT"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $NMEDIT" >&5 $as_echo "$NMEDIT" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_NMEDIT"; then ac_ct_NMEDIT=$NMEDIT # Extract the first word of "nmedit", so it can be a program name with args. set dummy nmedit; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_NMEDIT+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_NMEDIT"; then ac_cv_prog_ac_ct_NMEDIT="$ac_ct_NMEDIT" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_NMEDIT="nmedit" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_NMEDIT=$ac_cv_prog_ac_ct_NMEDIT if test -n "$ac_ct_NMEDIT"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_NMEDIT" >&5 $as_echo "$ac_ct_NMEDIT" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_NMEDIT" = x; then NMEDIT=":" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac NMEDIT=$ac_ct_NMEDIT fi else NMEDIT="$ac_cv_prog_NMEDIT" fi if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}lipo", so it can be a program name with args. set dummy ${ac_tool_prefix}lipo; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_LIPO+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$LIPO"; then ac_cv_prog_LIPO="$LIPO" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_LIPO="${ac_tool_prefix}lipo" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi LIPO=$ac_cv_prog_LIPO if test -n "$LIPO"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $LIPO" >&5 $as_echo "$LIPO" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_LIPO"; then ac_ct_LIPO=$LIPO # Extract the first word of "lipo", so it can be a program name with args. set dummy lipo; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_LIPO+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_LIPO"; then ac_cv_prog_ac_ct_LIPO="$ac_ct_LIPO" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_LIPO="lipo" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_LIPO=$ac_cv_prog_ac_ct_LIPO if test -n "$ac_ct_LIPO"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_LIPO" >&5 $as_echo "$ac_ct_LIPO" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_LIPO" = x; then LIPO=":" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac LIPO=$ac_ct_LIPO fi else LIPO="$ac_cv_prog_LIPO" fi if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}otool", so it can be a program name with args. set dummy ${ac_tool_prefix}otool; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_OTOOL+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$OTOOL"; then ac_cv_prog_OTOOL="$OTOOL" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_OTOOL="${ac_tool_prefix}otool" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi OTOOL=$ac_cv_prog_OTOOL if test -n "$OTOOL"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $OTOOL" >&5 $as_echo "$OTOOL" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_OTOOL"; then ac_ct_OTOOL=$OTOOL # Extract the first word of "otool", so it can be a program name with args. set dummy otool; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_OTOOL+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_OTOOL"; then ac_cv_prog_ac_ct_OTOOL="$ac_ct_OTOOL" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_OTOOL="otool" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_OTOOL=$ac_cv_prog_ac_ct_OTOOL if test -n "$ac_ct_OTOOL"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_OTOOL" >&5 $as_echo "$ac_ct_OTOOL" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_OTOOL" = x; then OTOOL=":" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac OTOOL=$ac_ct_OTOOL fi else OTOOL="$ac_cv_prog_OTOOL" fi if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}otool64", so it can be a program name with args. set dummy ${ac_tool_prefix}otool64; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_OTOOL64+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$OTOOL64"; then ac_cv_prog_OTOOL64="$OTOOL64" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_OTOOL64="${ac_tool_prefix}otool64" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi OTOOL64=$ac_cv_prog_OTOOL64 if test -n "$OTOOL64"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $OTOOL64" >&5 $as_echo "$OTOOL64" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_OTOOL64"; then ac_ct_OTOOL64=$OTOOL64 # Extract the first word of "otool64", so it can be a program name with args. set dummy otool64; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_OTOOL64+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_OTOOL64"; then ac_cv_prog_ac_ct_OTOOL64="$ac_ct_OTOOL64" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_OTOOL64="otool64" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_OTOOL64=$ac_cv_prog_ac_ct_OTOOL64 if test -n "$ac_ct_OTOOL64"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_OTOOL64" >&5 $as_echo "$ac_ct_OTOOL64" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_OTOOL64" = x; then OTOOL64=":" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac OTOOL64=$ac_ct_OTOOL64 fi else OTOOL64="$ac_cv_prog_OTOOL64" fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for -single_module linker flag" >&5 $as_echo_n "checking for -single_module linker flag... " >&6; } if ${lt_cv_apple_cc_single_mod+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_apple_cc_single_mod=no if test -z "$LT_MULTI_MODULE"; then # By default we will add the -single_module flag. You can override # by either setting the environment variable LT_MULTI_MODULE # non-empty at configure time, or by adding -multi_module to the # link flags. rm -rf libconftest.dylib* echo "int foo(void){return 1;}" > conftest.c echo "$LTCC $LTCFLAGS $LDFLAGS -o libconftest.dylib \ -dynamiclib -Wl,-single_module conftest.c" >&5 $LTCC $LTCFLAGS $LDFLAGS -o libconftest.dylib \ -dynamiclib -Wl,-single_module conftest.c 2>conftest.err _lt_result=$? # If there is a non-empty error log, and "single_module" # appears in it, assume the flag caused a linker warning if test -s conftest.err && $GREP single_module conftest.err; then cat conftest.err >&5 # Otherwise, if the output was created with a 0 exit code from # the compiler, it worked. elif test -f libconftest.dylib && test 0 = "$_lt_result"; then lt_cv_apple_cc_single_mod=yes else cat conftest.err >&5 fi rm -rf libconftest.dylib* rm -f conftest.* fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_apple_cc_single_mod" >&5 $as_echo "$lt_cv_apple_cc_single_mod" >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: checking for -exported_symbols_list linker flag" >&5 $as_echo_n "checking for -exported_symbols_list linker flag... " >&6; } if ${lt_cv_ld_exported_symbols_list+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_ld_exported_symbols_list=no save_LDFLAGS=$LDFLAGS echo "_main" > conftest.sym LDFLAGS="$LDFLAGS -Wl,-exported_symbols_list,conftest.sym" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : lt_cv_ld_exported_symbols_list=yes else lt_cv_ld_exported_symbols_list=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LDFLAGS=$save_LDFLAGS fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_ld_exported_symbols_list" >&5 $as_echo "$lt_cv_ld_exported_symbols_list" >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: checking for -force_load linker flag" >&5 $as_echo_n "checking for -force_load linker flag... " >&6; } if ${lt_cv_ld_force_load+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_ld_force_load=no cat > conftest.c << _LT_EOF int forced_loaded() { return 2;} _LT_EOF echo "$LTCC $LTCFLAGS -c -o conftest.o conftest.c" >&5 $LTCC $LTCFLAGS -c -o conftest.o conftest.c 2>&5 echo "$AR cru libconftest.a conftest.o" >&5 $AR cru libconftest.a conftest.o 2>&5 echo "$RANLIB libconftest.a" >&5 $RANLIB libconftest.a 2>&5 cat > conftest.c << _LT_EOF int main() { return 0;} _LT_EOF echo "$LTCC $LTCFLAGS $LDFLAGS -o conftest conftest.c -Wl,-force_load,./libconftest.a" >&5 $LTCC $LTCFLAGS $LDFLAGS -o conftest conftest.c -Wl,-force_load,./libconftest.a 2>conftest.err _lt_result=$? if test -s conftest.err && $GREP force_load conftest.err; then cat conftest.err >&5 elif test -f conftest && test 0 = "$_lt_result" && $GREP forced_load conftest >/dev/null 2>&1; then lt_cv_ld_force_load=yes else cat conftest.err >&5 fi rm -f conftest.err libconftest.a conftest conftest.c rm -rf conftest.dSYM fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_ld_force_load" >&5 $as_echo "$lt_cv_ld_force_load" >&6; } case $host_os in rhapsody* | darwin1.[012]) _lt_dar_allow_undefined='$wl-undefined ${wl}suppress' ;; darwin1.*) _lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;; darwin*) # darwin 5.x on # if running on 10.5 or later, the deployment target defaults # to the OS version, if on x86, and 10.4, the deployment # target defaults to 10.4. Don't you love it? case ${MACOSX_DEPLOYMENT_TARGET-10.0},$host in 10.0,*86*-darwin8*|10.0,*-darwin[91]*) _lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;; 10.[012][,.]*) _lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;; 10.*) _lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;; esac ;; esac if test yes = "$lt_cv_apple_cc_single_mod"; then _lt_dar_single_mod='$single_module' fi if test yes = "$lt_cv_ld_exported_symbols_list"; then _lt_dar_export_syms=' $wl-exported_symbols_list,$output_objdir/$libname-symbols.expsym' else _lt_dar_export_syms='~$NMEDIT -s $output_objdir/$libname-symbols.expsym $lib' fi if test : != "$DSYMUTIL" && test no = "$lt_cv_ld_force_load"; then _lt_dsymutil='~$DSYMUTIL $lib || :' else _lt_dsymutil= fi ;; esac # func_munge_path_list VARIABLE PATH # ----------------------------------- # VARIABLE is name of variable containing _space_ separated list of # directories to be munged by the contents of PATH, which is string # having a format: # "DIR[:DIR]:" # string "DIR[ DIR]" will be prepended to VARIABLE # ":DIR[:DIR]" # string "DIR[ DIR]" will be appended to VARIABLE # "DIRP[:DIRP]::[DIRA:]DIRA" # string "DIRP[ DIRP]" will be prepended to VARIABLE and string # "DIRA[ DIRA]" will be appended to VARIABLE # "DIR[:DIR]" # VARIABLE will be replaced by "DIR[ DIR]" func_munge_path_list () { case x$2 in x) ;; *:) eval $1=\"`$ECHO $2 | $SED 's/:/ /g'` \$$1\" ;; x:*) eval $1=\"\$$1 `$ECHO $2 | $SED 's/:/ /g'`\" ;; *::*) eval $1=\"\$$1\ `$ECHO $2 | $SED -e 's/.*:://' -e 's/:/ /g'`\" eval $1=\"`$ECHO $2 | $SED -e 's/::.*//' -e 's/:/ /g'`\ \$$1\" ;; *) eval $1=\"`$ECHO $2 | $SED 's/:/ /g'`\" ;; esac } ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu { $as_echo "$as_me:${as_lineno-$LINENO}: checking how to run the C preprocessor" >&5 $as_echo_n "checking how to run the C preprocessor... " >&6; } # On Suns, sometimes $CPP names a directory. if test -n "$CPP" && test -d "$CPP"; then CPP= fi if test -z "$CPP"; then if ${ac_cv_prog_CPP+:} false; then : $as_echo_n "(cached) " >&6 else # Double quotes because CPP needs to be expanded for CPP in "$CC -E" "$CC -E -traditional-cpp" "/lib/cpp" do ac_preproc_ok=false for ac_c_preproc_warn_flag in '' yes do # Use a header file that comes with gcc, so configuring glibc # with a fresh cross-compiler works. # Prefer to if __STDC__ is defined, since # exists even on freestanding compilers. # On the NeXT, cc -E runs the code through the compiler's parser, # not just through cpp. "Syntax error" is here to catch this case. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #ifdef __STDC__ # include #else # include #endif Syntax error _ACEOF if ac_fn_c_try_cpp "$LINENO"; then : else # Broken: fails on valid input. continue fi rm -f conftest.err conftest.i conftest.$ac_ext # OK, works on sane cases. Now check whether nonexistent headers # can be detected and how. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include _ACEOF if ac_fn_c_try_cpp "$LINENO"; then : # Broken: success on invalid input. continue else # Passes both tests. ac_preproc_ok=: break fi rm -f conftest.err conftest.i conftest.$ac_ext done # Because of `break', _AC_PREPROC_IFELSE's cleaning code was skipped. rm -f conftest.i conftest.err conftest.$ac_ext if $ac_preproc_ok; then : break fi done ac_cv_prog_CPP=$CPP fi CPP=$ac_cv_prog_CPP else ac_cv_prog_CPP=$CPP fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $CPP" >&5 $as_echo "$CPP" >&6; } ac_preproc_ok=false for ac_c_preproc_warn_flag in '' yes do # Use a header file that comes with gcc, so configuring glibc # with a fresh cross-compiler works. # Prefer to if __STDC__ is defined, since # exists even on freestanding compilers. # On the NeXT, cc -E runs the code through the compiler's parser, # not just through cpp. "Syntax error" is here to catch this case. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #ifdef __STDC__ # include #else # include #endif Syntax error _ACEOF if ac_fn_c_try_cpp "$LINENO"; then : else # Broken: fails on valid input. continue fi rm -f conftest.err conftest.i conftest.$ac_ext # OK, works on sane cases. Now check whether nonexistent headers # can be detected and how. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include _ACEOF if ac_fn_c_try_cpp "$LINENO"; then : # Broken: success on invalid input. continue else # Passes both tests. ac_preproc_ok=: break fi rm -f conftest.err conftest.i conftest.$ac_ext done # Because of `break', _AC_PREPROC_IFELSE's cleaning code was skipped. rm -f conftest.i conftest.err conftest.$ac_ext if $ac_preproc_ok; then : else { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 $as_echo "$as_me: error: in \`$ac_pwd':" >&2;} as_fn_error $? "C preprocessor \"$CPP\" fails sanity check See \`config.log' for more details" "$LINENO" 5; } fi ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu { $as_echo "$as_me:${as_lineno-$LINENO}: checking for ANSI C header files" >&5 $as_echo_n "checking for ANSI C header files... " >&6; } if ${ac_cv_header_stdc+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include #include #include #include int main () { ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_cv_header_stdc=yes else ac_cv_header_stdc=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext if test $ac_cv_header_stdc = yes; then # SunOS 4.x string.h does not declare mem*, contrary to ANSI. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include _ACEOF if (eval "$ac_cpp conftest.$ac_ext") 2>&5 | $EGREP "memchr" >/dev/null 2>&1; then : else ac_cv_header_stdc=no fi rm -f conftest* fi if test $ac_cv_header_stdc = yes; then # ISC 2.0.2 stdlib.h does not declare free, contrary to ANSI. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include _ACEOF if (eval "$ac_cpp conftest.$ac_ext") 2>&5 | $EGREP "free" >/dev/null 2>&1; then : else ac_cv_header_stdc=no fi rm -f conftest* fi if test $ac_cv_header_stdc = yes; then # /bin/cc in Irix-4.0.5 gets non-ANSI ctype macros unless using -ansi. if test "$cross_compiling" = yes; then : : else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include #include #if ((' ' & 0x0FF) == 0x020) # define ISLOWER(c) ('a' <= (c) && (c) <= 'z') # define TOUPPER(c) (ISLOWER(c) ? 'A' + ((c) - 'a') : (c)) #else # define ISLOWER(c) \ (('a' <= (c) && (c) <= 'i') \ || ('j' <= (c) && (c) <= 'r') \ || ('s' <= (c) && (c) <= 'z')) # define TOUPPER(c) (ISLOWER(c) ? ((c) | 0x40) : (c)) #endif #define XOR(e, f) (((e) && !(f)) || (!(e) && (f))) int main () { int i; for (i = 0; i < 256; i++) if (XOR (islower (i), ISLOWER (i)) || toupper (i) != TOUPPER (i)) return 2; return 0; } _ACEOF if ac_fn_c_try_run "$LINENO"; then : else ac_cv_header_stdc=no fi rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \ conftest.$ac_objext conftest.beam conftest.$ac_ext fi fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_header_stdc" >&5 $as_echo "$ac_cv_header_stdc" >&6; } if test $ac_cv_header_stdc = yes; then $as_echo "#define STDC_HEADERS 1" >>confdefs.h fi # On IRIX 5.3, sys/types and inttypes.h are conflicting. for ac_header in sys/types.h sys/stat.h stdlib.h string.h memory.h strings.h \ inttypes.h stdint.h unistd.h do : as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh` ac_fn_c_check_header_compile "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default " if eval test \"x\$"$as_ac_Header"\" = x"yes"; then : cat >>confdefs.h <<_ACEOF #define `$as_echo "HAVE_$ac_header" | $as_tr_cpp` 1 _ACEOF fi done for ac_header in dlfcn.h do : ac_fn_c_check_header_compile "$LINENO" "dlfcn.h" "ac_cv_header_dlfcn_h" "$ac_includes_default " if test "x$ac_cv_header_dlfcn_h" = xyes; then : cat >>confdefs.h <<_ACEOF #define HAVE_DLFCN_H 1 _ACEOF fi done func_stripname_cnf () { case $2 in .*) func_stripname_result=`$ECHO "$3" | $SED "s%^$1%%; s%\\\\$2\$%%"`;; *) func_stripname_result=`$ECHO "$3" | $SED "s%^$1%%; s%$2\$%%"`;; esac } # func_stripname_cnf # Set options enable_dlopen=no enable_win32_dll=no # Check whether --enable-shared was given. if test "${enable_shared+set}" = set; then : enableval=$enable_shared; p=${PACKAGE-default} case $enableval in yes) enable_shared=yes ;; no) enable_shared=no ;; *) enable_shared=no # Look at the argument we got. We use all the common list separators. lt_save_ifs=$IFS; IFS=$IFS$PATH_SEPARATOR, for pkg in $enableval; do IFS=$lt_save_ifs if test "X$pkg" = "X$p"; then enable_shared=yes fi done IFS=$lt_save_ifs ;; esac else enable_shared=yes fi # Check whether --enable-static was given. if test "${enable_static+set}" = set; then : enableval=$enable_static; p=${PACKAGE-default} case $enableval in yes) enable_static=yes ;; no) enable_static=no ;; *) enable_static=no # Look at the argument we got. We use all the common list separators. lt_save_ifs=$IFS; IFS=$IFS$PATH_SEPARATOR, for pkg in $enableval; do IFS=$lt_save_ifs if test "X$pkg" = "X$p"; then enable_static=yes fi done IFS=$lt_save_ifs ;; esac else enable_static=yes fi # Check whether --with-pic was given. if test "${with_pic+set}" = set; then : withval=$with_pic; lt_p=${PACKAGE-default} case $withval in yes|no) pic_mode=$withval ;; *) pic_mode=default # Look at the argument we got. We use all the common list separators. lt_save_ifs=$IFS; IFS=$IFS$PATH_SEPARATOR, for lt_pkg in $withval; do IFS=$lt_save_ifs if test "X$lt_pkg" = "X$lt_p"; then pic_mode=yes fi done IFS=$lt_save_ifs ;; esac else pic_mode=default fi # Check whether --enable-fast-install was given. if test "${enable_fast_install+set}" = set; then : enableval=$enable_fast_install; p=${PACKAGE-default} case $enableval in yes) enable_fast_install=yes ;; no) enable_fast_install=no ;; *) enable_fast_install=no # Look at the argument we got. We use all the common list separators. lt_save_ifs=$IFS; IFS=$IFS$PATH_SEPARATOR, for pkg in $enableval; do IFS=$lt_save_ifs if test "X$pkg" = "X$p"; then enable_fast_install=yes fi done IFS=$lt_save_ifs ;; esac else enable_fast_install=yes fi shared_archive_member_spec= case $host,$enable_shared in power*-*-aix[5-9]*,yes) { $as_echo "$as_me:${as_lineno-$LINENO}: checking which variant of shared library versioning to provide" >&5 $as_echo_n "checking which variant of shared library versioning to provide... " >&6; } # Check whether --with-aix-soname was given. if test "${with_aix_soname+set}" = set; then : withval=$with_aix_soname; case $withval in aix|svr4|both) ;; *) as_fn_error $? "Unknown argument to --with-aix-soname" "$LINENO" 5 ;; esac lt_cv_with_aix_soname=$with_aix_soname else if ${lt_cv_with_aix_soname+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_with_aix_soname=aix fi with_aix_soname=$lt_cv_with_aix_soname fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_aix_soname" >&5 $as_echo "$with_aix_soname" >&6; } if test aix != "$with_aix_soname"; then # For the AIX way of multilib, we name the shared archive member # based on the bitwidth used, traditionally 'shr.o' or 'shr_64.o', # and 'shr.imp' or 'shr_64.imp', respectively, for the Import File. # Even when GNU compilers ignore OBJECT_MODE but need '-maix64' flag, # the AIX toolchain works better with OBJECT_MODE set (default 32). if test 64 = "${OBJECT_MODE-32}"; then shared_archive_member_spec=shr_64 else shared_archive_member_spec=shr fi fi ;; *) with_aix_soname=aix ;; esac # This can be used to rebuild libtool when needed LIBTOOL_DEPS=$ltmain # Always use our own libtool. LIBTOOL='$(SHELL) $(top_builddir)/libtool' test -z "$LN_S" && LN_S="ln -s" if test -n "${ZSH_VERSION+set}"; then setopt NO_GLOB_SUBST fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for objdir" >&5 $as_echo_n "checking for objdir... " >&6; } if ${lt_cv_objdir+:} false; then : $as_echo_n "(cached) " >&6 else rm -f .libs 2>/dev/null mkdir .libs 2>/dev/null if test -d .libs; then lt_cv_objdir=.libs else # MS-DOS does not allow filenames that begin with a dot. lt_cv_objdir=_libs fi rmdir .libs 2>/dev/null fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_objdir" >&5 $as_echo "$lt_cv_objdir" >&6; } objdir=$lt_cv_objdir cat >>confdefs.h <<_ACEOF #define LT_OBJDIR "$lt_cv_objdir/" _ACEOF case $host_os in aix3*) # AIX sometimes has problems with the GCC collect2 program. For some # reason, if we set the COLLECT_NAMES environment variable, the problems # vanish in a puff of smoke. if test set != "${COLLECT_NAMES+set}"; then COLLECT_NAMES= export COLLECT_NAMES fi ;; esac # Global variables: ofile=libtool can_build_shared=yes # All known linkers require a '.a' archive for static linking (except MSVC, # which needs '.lib'). libext=a with_gnu_ld=$lt_cv_prog_gnu_ld old_CC=$CC old_CFLAGS=$CFLAGS # Set sane defaults for various variables test -z "$CC" && CC=cc test -z "$LTCC" && LTCC=$CC test -z "$LTCFLAGS" && LTCFLAGS=$CFLAGS test -z "$LD" && LD=ld test -z "$ac_objext" && ac_objext=o func_cc_basename $compiler cc_basename=$func_cc_basename_result # Only perform the check for file, if the check method requires it test -z "$MAGIC_CMD" && MAGIC_CMD=file case $deplibs_check_method in file_magic*) if test "$file_magic_cmd" = '$MAGIC_CMD'; then { $as_echo "$as_me:${as_lineno-$LINENO}: checking for ${ac_tool_prefix}file" >&5 $as_echo_n "checking for ${ac_tool_prefix}file... " >&6; } if ${lt_cv_path_MAGIC_CMD+:} false; then : $as_echo_n "(cached) " >&6 else case $MAGIC_CMD in [\\/*] | ?:[\\/]*) lt_cv_path_MAGIC_CMD=$MAGIC_CMD # Let the user override the test with a path. ;; *) lt_save_MAGIC_CMD=$MAGIC_CMD lt_save_ifs=$IFS; IFS=$PATH_SEPARATOR ac_dummy="/usr/bin$PATH_SEPARATOR$PATH" for ac_dir in $ac_dummy; do IFS=$lt_save_ifs test -z "$ac_dir" && ac_dir=. if test -f "$ac_dir/${ac_tool_prefix}file"; then lt_cv_path_MAGIC_CMD=$ac_dir/"${ac_tool_prefix}file" if test -n "$file_magic_test_file"; then case $deplibs_check_method in "file_magic "*) file_magic_regex=`expr "$deplibs_check_method" : "file_magic \(.*\)"` MAGIC_CMD=$lt_cv_path_MAGIC_CMD if eval $file_magic_cmd \$file_magic_test_file 2> /dev/null | $EGREP "$file_magic_regex" > /dev/null; then : else cat <<_LT_EOF 1>&2 *** Warning: the command libtool uses to detect shared libraries, *** $file_magic_cmd, produces output that libtool cannot recognize. *** The result is that libtool may fail to recognize shared libraries *** as such. This will affect the creation of libtool libraries that *** depend on shared libraries, but programs linked with such libtool *** libraries will work regardless of this problem. Nevertheless, you *** may want to report the problem to your system manager and/or to *** bug-libtool@gnu.org _LT_EOF fi ;; esac fi break fi done IFS=$lt_save_ifs MAGIC_CMD=$lt_save_MAGIC_CMD ;; esac fi MAGIC_CMD=$lt_cv_path_MAGIC_CMD if test -n "$MAGIC_CMD"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $MAGIC_CMD" >&5 $as_echo "$MAGIC_CMD" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test -z "$lt_cv_path_MAGIC_CMD"; then if test -n "$ac_tool_prefix"; then { $as_echo "$as_me:${as_lineno-$LINENO}: checking for file" >&5 $as_echo_n "checking for file... " >&6; } if ${lt_cv_path_MAGIC_CMD+:} false; then : $as_echo_n "(cached) " >&6 else case $MAGIC_CMD in [\\/*] | ?:[\\/]*) lt_cv_path_MAGIC_CMD=$MAGIC_CMD # Let the user override the test with a path. ;; *) lt_save_MAGIC_CMD=$MAGIC_CMD lt_save_ifs=$IFS; IFS=$PATH_SEPARATOR ac_dummy="/usr/bin$PATH_SEPARATOR$PATH" for ac_dir in $ac_dummy; do IFS=$lt_save_ifs test -z "$ac_dir" && ac_dir=. if test -f "$ac_dir/file"; then lt_cv_path_MAGIC_CMD=$ac_dir/"file" if test -n "$file_magic_test_file"; then case $deplibs_check_method in "file_magic "*) file_magic_regex=`expr "$deplibs_check_method" : "file_magic \(.*\)"` MAGIC_CMD=$lt_cv_path_MAGIC_CMD if eval $file_magic_cmd \$file_magic_test_file 2> /dev/null | $EGREP "$file_magic_regex" > /dev/null; then : else cat <<_LT_EOF 1>&2 *** Warning: the command libtool uses to detect shared libraries, *** $file_magic_cmd, produces output that libtool cannot recognize. *** The result is that libtool may fail to recognize shared libraries *** as such. This will affect the creation of libtool libraries that *** depend on shared libraries, but programs linked with such libtool *** libraries will work regardless of this problem. Nevertheless, you *** may want to report the problem to your system manager and/or to *** bug-libtool@gnu.org _LT_EOF fi ;; esac fi break fi done IFS=$lt_save_ifs MAGIC_CMD=$lt_save_MAGIC_CMD ;; esac fi MAGIC_CMD=$lt_cv_path_MAGIC_CMD if test -n "$MAGIC_CMD"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $MAGIC_CMD" >&5 $as_echo "$MAGIC_CMD" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi else MAGIC_CMD=: fi fi fi ;; esac # Use C for the default configuration in the libtool script lt_save_CC=$CC ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu # Source file extension for C test sources. ac_ext=c # Object file extension for compiled C test sources. objext=o objext=$objext # Code to be used in simple compile tests lt_simple_compile_test_code="int some_variable = 0;" # Code to be used in simple link tests lt_simple_link_test_code='int main(){return(0);}' # If no C compiler was specified, use CC. LTCC=${LTCC-"$CC"} # If no C compiler flags were specified, use CFLAGS. LTCFLAGS=${LTCFLAGS-"$CFLAGS"} # Allow CC to be a program name with arguments. compiler=$CC # Save the default compiler, since it gets overwritten when the other # tags are being tested, and _LT_TAGVAR(compiler, []) is a NOP. compiler_DEFAULT=$CC # save warnings/boilerplate of simple test code ac_outfile=conftest.$ac_objext echo "$lt_simple_compile_test_code" >conftest.$ac_ext eval "$ac_compile" 2>&1 >/dev/null | $SED '/^$/d; /^ *+/d' >conftest.err _lt_compiler_boilerplate=`cat conftest.err` $RM conftest* ac_outfile=conftest.$ac_objext echo "$lt_simple_link_test_code" >conftest.$ac_ext eval "$ac_link" 2>&1 >/dev/null | $SED '/^$/d; /^ *+/d' >conftest.err _lt_linker_boilerplate=`cat conftest.err` $RM -r conftest* ## CAVEAT EMPTOR: ## There is no encapsulation within the following macros, do not change ## the running order or otherwise move them around unless you know exactly ## what you are doing... if test -n "$compiler"; then lt_prog_compiler_no_builtin_flag= if test yes = "$GCC"; then case $cc_basename in nvcc*) lt_prog_compiler_no_builtin_flag=' -Xcompiler -fno-builtin' ;; *) lt_prog_compiler_no_builtin_flag=' -fno-builtin' ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $compiler supports -fno-rtti -fno-exceptions" >&5 $as_echo_n "checking if $compiler supports -fno-rtti -fno-exceptions... " >&6; } if ${lt_cv_prog_compiler_rtti_exceptions+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_prog_compiler_rtti_exceptions=no ac_outfile=conftest.$ac_objext echo "$lt_simple_compile_test_code" > conftest.$ac_ext lt_compiler_flag="-fno-rtti -fno-exceptions" ## exclude from sc_useless_quotes_in_assignment # Insert the option either (1) after the last *FLAGS variable, or # (2) before a word containing "conftest.", or (3) at the end. # Note that $ac_compile itself does not contain backslashes and begins # with a dollar sign (not a hyphen), so the echo should work correctly. # The option is referenced via a variable to avoid confusing sed. lt_compile=`echo "$ac_compile" | $SED \ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` (eval echo "\"\$as_me:$LINENO: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings other than the usual output. $ECHO "$_lt_compiler_boilerplate" | $SED '/^$/d' >conftest.exp $SED '/^$/d; /^ *+/d' conftest.err >conftest.er2 if test ! -s conftest.er2 || diff conftest.exp conftest.er2 >/dev/null; then lt_cv_prog_compiler_rtti_exceptions=yes fi fi $RM conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_rtti_exceptions" >&5 $as_echo "$lt_cv_prog_compiler_rtti_exceptions" >&6; } if test yes = "$lt_cv_prog_compiler_rtti_exceptions"; then lt_prog_compiler_no_builtin_flag="$lt_prog_compiler_no_builtin_flag -fno-rtti -fno-exceptions" else : fi fi lt_prog_compiler_wl= lt_prog_compiler_pic= lt_prog_compiler_static= if test yes = "$GCC"; then lt_prog_compiler_wl='-Wl,' lt_prog_compiler_static='-static' case $host_os in aix*) # All AIX code is PIC. if test ia64 = "$host_cpu"; then # AIX 5 now supports IA64 processor lt_prog_compiler_static='-Bstatic' fi lt_prog_compiler_pic='-fPIC' ;; amigaos*) case $host_cpu in powerpc) # see comment about AmigaOS4 .so support lt_prog_compiler_pic='-fPIC' ;; m68k) # FIXME: we need at least 68020 code to build shared libraries, but # adding the '-m68020' flag to GCC prevents building anything better, # like '-m68040'. lt_prog_compiler_pic='-m68020 -resident32 -malways-restore-a4' ;; esac ;; beos* | irix5* | irix6* | nonstopux* | osf3* | osf4* | osf5*) # PIC is the default for these OSes. ;; mingw* | cygwin* | pw32* | os2* | cegcc*) # This hack is so that the source file can tell whether it is being # built for inclusion in a dll (and should export symbols for example). # Although the cygwin gcc ignores -fPIC, still need this for old-style # (--disable-auto-import) libraries lt_prog_compiler_pic='-DDLL_EXPORT' case $host_os in os2*) lt_prog_compiler_static='$wl-static' ;; esac ;; darwin* | rhapsody*) # PIC is the default on this platform # Common symbols not allowed in MH_DYLIB files lt_prog_compiler_pic='-fno-common' ;; haiku*) # PIC is the default for Haiku. # The "-static" flag exists, but is broken. lt_prog_compiler_static= ;; hpux*) # PIC is the default for 64-bit PA HP-UX, but not for 32-bit # PA HP-UX. On IA64 HP-UX, PIC is the default but the pic flag # sets the default TLS model and affects inlining. case $host_cpu in hppa*64*) # +Z the default ;; *) lt_prog_compiler_pic='-fPIC' ;; esac ;; interix[3-9]*) # Interix 3.x gcc -fpic/-fPIC options generate broken code. # Instead, we relocate shared libraries at runtime. ;; msdosdjgpp*) # Just because we use GCC doesn't mean we suddenly get shared libraries # on systems that don't support them. lt_prog_compiler_can_build_shared=no enable_shared=no ;; *nto* | *qnx*) # QNX uses GNU C++, but need to define -shared option too, otherwise # it will coredump. lt_prog_compiler_pic='-fPIC -shared' ;; sysv4*MP*) if test -d /usr/nec; then lt_prog_compiler_pic=-Kconform_pic fi ;; *) lt_prog_compiler_pic='-fPIC' ;; esac case $cc_basename in nvcc*) # Cuda Compiler Driver 2.2 lt_prog_compiler_wl='-Xlinker ' if test -n "$lt_prog_compiler_pic"; then lt_prog_compiler_pic="-Xcompiler $lt_prog_compiler_pic" fi ;; esac else # PORTME Check for flag to pass linker flags through the system compiler. case $host_os in aix*) lt_prog_compiler_wl='-Wl,' if test ia64 = "$host_cpu"; then # AIX 5 now supports IA64 processor lt_prog_compiler_static='-Bstatic' else lt_prog_compiler_static='-bnso -bI:/lib/syscalls.exp' fi ;; darwin* | rhapsody*) # PIC is the default on this platform # Common symbols not allowed in MH_DYLIB files lt_prog_compiler_pic='-fno-common' case $cc_basename in nagfor*) # NAG Fortran compiler lt_prog_compiler_wl='-Wl,-Wl,,' lt_prog_compiler_pic='-PIC' lt_prog_compiler_static='-Bstatic' ;; esac ;; mingw* | cygwin* | pw32* | os2* | cegcc*) # This hack is so that the source file can tell whether it is being # built for inclusion in a dll (and should export symbols for example). lt_prog_compiler_pic='-DDLL_EXPORT' case $host_os in os2*) lt_prog_compiler_static='$wl-static' ;; esac ;; hpux9* | hpux10* | hpux11*) lt_prog_compiler_wl='-Wl,' # PIC is the default for IA64 HP-UX and 64-bit HP-UX, but # not for PA HP-UX. case $host_cpu in hppa*64*|ia64*) # +Z the default ;; *) lt_prog_compiler_pic='+Z' ;; esac # Is there a better lt_prog_compiler_static that works with the bundled CC? lt_prog_compiler_static='$wl-a ${wl}archive' ;; irix5* | irix6* | nonstopux*) lt_prog_compiler_wl='-Wl,' # PIC (with -KPIC) is the default. lt_prog_compiler_static='-non_shared' ;; linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*) case $cc_basename in # old Intel for x86_64, which still supported -KPIC. ecc*) lt_prog_compiler_wl='-Wl,' lt_prog_compiler_pic='-KPIC' lt_prog_compiler_static='-static' ;; # icc used to be incompatible with GCC. # ICC 10 doesn't accept -KPIC any more. icc* | ifort*) lt_prog_compiler_wl='-Wl,' lt_prog_compiler_pic='-fPIC' lt_prog_compiler_static='-static' ;; # Lahey Fortran 8.1. lf95*) lt_prog_compiler_wl='-Wl,' lt_prog_compiler_pic='--shared' lt_prog_compiler_static='--static' ;; nagfor*) # NAG Fortran compiler lt_prog_compiler_wl='-Wl,-Wl,,' lt_prog_compiler_pic='-PIC' lt_prog_compiler_static='-Bstatic' ;; tcc*) # Fabrice Bellard et al's Tiny C Compiler lt_prog_compiler_wl='-Wl,' lt_prog_compiler_pic='-fPIC' lt_prog_compiler_static='-static' ;; pgcc* | pgf77* | pgf90* | pgf95* | pgfortran*) # Portland Group compilers (*not* the Pentium gcc compiler, # which looks to be a dead project) lt_prog_compiler_wl='-Wl,' lt_prog_compiler_pic='-fpic' lt_prog_compiler_static='-Bstatic' ;; ccc*) lt_prog_compiler_wl='-Wl,' # All Alpha code is PIC. lt_prog_compiler_static='-non_shared' ;; xl* | bgxl* | bgf* | mpixl*) # IBM XL C 8.0/Fortran 10.1, 11.1 on PPC and BlueGene lt_prog_compiler_wl='-Wl,' lt_prog_compiler_pic='-qpic' lt_prog_compiler_static='-qstaticlink' ;; *) case `$CC -V 2>&1 | sed 5q` in *Sun\ Ceres\ Fortran* | *Sun*Fortran*\ [1-7].* | *Sun*Fortran*\ 8.[0-3]*) # Sun Fortran 8.3 passes all unrecognized flags to the linker lt_prog_compiler_pic='-KPIC' lt_prog_compiler_static='-Bstatic' lt_prog_compiler_wl='' ;; *Sun\ F* | *Sun*Fortran*) lt_prog_compiler_pic='-KPIC' lt_prog_compiler_static='-Bstatic' lt_prog_compiler_wl='-Qoption ld ' ;; *Sun\ C*) # Sun C 5.9 lt_prog_compiler_pic='-KPIC' lt_prog_compiler_static='-Bstatic' lt_prog_compiler_wl='-Wl,' ;; *Intel*\ [CF]*Compiler*) lt_prog_compiler_wl='-Wl,' lt_prog_compiler_pic='-fPIC' lt_prog_compiler_static='-static' ;; *Portland\ Group*) lt_prog_compiler_wl='-Wl,' lt_prog_compiler_pic='-fpic' lt_prog_compiler_static='-Bstatic' ;; esac ;; esac ;; newsos6) lt_prog_compiler_pic='-KPIC' lt_prog_compiler_static='-Bstatic' ;; *nto* | *qnx*) # QNX uses GNU C++, but need to define -shared option too, otherwise # it will coredump. lt_prog_compiler_pic='-fPIC -shared' ;; osf3* | osf4* | osf5*) lt_prog_compiler_wl='-Wl,' # All OSF/1 code is PIC. lt_prog_compiler_static='-non_shared' ;; rdos*) lt_prog_compiler_static='-non_shared' ;; solaris*) lt_prog_compiler_pic='-KPIC' lt_prog_compiler_static='-Bstatic' case $cc_basename in f77* | f90* | f95* | sunf77* | sunf90* | sunf95*) lt_prog_compiler_wl='-Qoption ld ';; *) lt_prog_compiler_wl='-Wl,';; esac ;; sunos4*) lt_prog_compiler_wl='-Qoption ld ' lt_prog_compiler_pic='-PIC' lt_prog_compiler_static='-Bstatic' ;; sysv4 | sysv4.2uw2* | sysv4.3*) lt_prog_compiler_wl='-Wl,' lt_prog_compiler_pic='-KPIC' lt_prog_compiler_static='-Bstatic' ;; sysv4*MP*) if test -d /usr/nec; then lt_prog_compiler_pic='-Kconform_pic' lt_prog_compiler_static='-Bstatic' fi ;; sysv5* | unixware* | sco3.2v5* | sco5v6* | OpenUNIX*) lt_prog_compiler_wl='-Wl,' lt_prog_compiler_pic='-KPIC' lt_prog_compiler_static='-Bstatic' ;; unicos*) lt_prog_compiler_wl='-Wl,' lt_prog_compiler_can_build_shared=no ;; uts4*) lt_prog_compiler_pic='-pic' lt_prog_compiler_static='-Bstatic' ;; *) lt_prog_compiler_can_build_shared=no ;; esac fi case $host_os in # For platforms that do not support PIC, -DPIC is meaningless: *djgpp*) lt_prog_compiler_pic= ;; *) lt_prog_compiler_pic="$lt_prog_compiler_pic -DPIC" ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $compiler option to produce PIC" >&5 $as_echo_n "checking for $compiler option to produce PIC... " >&6; } if ${lt_cv_prog_compiler_pic+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_prog_compiler_pic=$lt_prog_compiler_pic fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_pic" >&5 $as_echo "$lt_cv_prog_compiler_pic" >&6; } lt_prog_compiler_pic=$lt_cv_prog_compiler_pic # # Check to make sure the PIC flag actually works. # if test -n "$lt_prog_compiler_pic"; then { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $compiler PIC flag $lt_prog_compiler_pic works" >&5 $as_echo_n "checking if $compiler PIC flag $lt_prog_compiler_pic works... " >&6; } if ${lt_cv_prog_compiler_pic_works+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_prog_compiler_pic_works=no ac_outfile=conftest.$ac_objext echo "$lt_simple_compile_test_code" > conftest.$ac_ext lt_compiler_flag="$lt_prog_compiler_pic -DPIC" ## exclude from sc_useless_quotes_in_assignment # Insert the option either (1) after the last *FLAGS variable, or # (2) before a word containing "conftest.", or (3) at the end. # Note that $ac_compile itself does not contain backslashes and begins # with a dollar sign (not a hyphen), so the echo should work correctly. # The option is referenced via a variable to avoid confusing sed. lt_compile=`echo "$ac_compile" | $SED \ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` (eval echo "\"\$as_me:$LINENO: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings other than the usual output. $ECHO "$_lt_compiler_boilerplate" | $SED '/^$/d' >conftest.exp $SED '/^$/d; /^ *+/d' conftest.err >conftest.er2 if test ! -s conftest.er2 || diff conftest.exp conftest.er2 >/dev/null; then lt_cv_prog_compiler_pic_works=yes fi fi $RM conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_pic_works" >&5 $as_echo "$lt_cv_prog_compiler_pic_works" >&6; } if test yes = "$lt_cv_prog_compiler_pic_works"; then case $lt_prog_compiler_pic in "" | " "*) ;; *) lt_prog_compiler_pic=" $lt_prog_compiler_pic" ;; esac else lt_prog_compiler_pic= lt_prog_compiler_can_build_shared=no fi fi # # Check to make sure the static flag actually works. # wl=$lt_prog_compiler_wl eval lt_tmp_static_flag=\"$lt_prog_compiler_static\" { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $compiler static flag $lt_tmp_static_flag works" >&5 $as_echo_n "checking if $compiler static flag $lt_tmp_static_flag works... " >&6; } if ${lt_cv_prog_compiler_static_works+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_prog_compiler_static_works=no save_LDFLAGS=$LDFLAGS LDFLAGS="$LDFLAGS $lt_tmp_static_flag" echo "$lt_simple_link_test_code" > conftest.$ac_ext if (eval $ac_link 2>conftest.err) && test -s conftest$ac_exeext; then # The linker can only warn and ignore the option if not recognized # So say no if there are warnings if test -s conftest.err; then # Append any errors to the config.log. cat conftest.err 1>&5 $ECHO "$_lt_linker_boilerplate" | $SED '/^$/d' > conftest.exp $SED '/^$/d; /^ *+/d' conftest.err >conftest.er2 if diff conftest.exp conftest.er2 >/dev/null; then lt_cv_prog_compiler_static_works=yes fi else lt_cv_prog_compiler_static_works=yes fi fi $RM -r conftest* LDFLAGS=$save_LDFLAGS fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_static_works" >&5 $as_echo "$lt_cv_prog_compiler_static_works" >&6; } if test yes = "$lt_cv_prog_compiler_static_works"; then : else lt_prog_compiler_static= fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $compiler supports -c -o file.$ac_objext" >&5 $as_echo_n "checking if $compiler supports -c -o file.$ac_objext... " >&6; } if ${lt_cv_prog_compiler_c_o+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_prog_compiler_c_o=no $RM -r conftest 2>/dev/null mkdir conftest cd conftest mkdir out echo "$lt_simple_compile_test_code" > conftest.$ac_ext lt_compiler_flag="-o out/conftest2.$ac_objext" # Insert the option either (1) after the last *FLAGS variable, or # (2) before a word containing "conftest.", or (3) at the end. # Note that $ac_compile itself does not contain backslashes and begins # with a dollar sign (not a hyphen), so the echo should work correctly. lt_compile=`echo "$ac_compile" | $SED \ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` (eval echo "\"\$as_me:$LINENO: $lt_compile\"" >&5) (eval "$lt_compile" 2>out/conftest.err) ac_status=$? cat out/conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 if (exit $ac_status) && test -s out/conftest2.$ac_objext then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings $ECHO "$_lt_compiler_boilerplate" | $SED '/^$/d' > out/conftest.exp $SED '/^$/d; /^ *+/d' out/conftest.err >out/conftest.er2 if test ! -s out/conftest.er2 || diff out/conftest.exp out/conftest.er2 >/dev/null; then lt_cv_prog_compiler_c_o=yes fi fi chmod u+w . 2>&5 $RM conftest* # SGI C++ compiler will create directory out/ii_files/ for # template instantiation test -d out/ii_files && $RM out/ii_files/* && rmdir out/ii_files $RM out/* && rmdir out cd .. $RM -r conftest $RM conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_c_o" >&5 $as_echo "$lt_cv_prog_compiler_c_o" >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $compiler supports -c -o file.$ac_objext" >&5 $as_echo_n "checking if $compiler supports -c -o file.$ac_objext... " >&6; } if ${lt_cv_prog_compiler_c_o+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_prog_compiler_c_o=no $RM -r conftest 2>/dev/null mkdir conftest cd conftest mkdir out echo "$lt_simple_compile_test_code" > conftest.$ac_ext lt_compiler_flag="-o out/conftest2.$ac_objext" # Insert the option either (1) after the last *FLAGS variable, or # (2) before a word containing "conftest.", or (3) at the end. # Note that $ac_compile itself does not contain backslashes and begins # with a dollar sign (not a hyphen), so the echo should work correctly. lt_compile=`echo "$ac_compile" | $SED \ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` (eval echo "\"\$as_me:$LINENO: $lt_compile\"" >&5) (eval "$lt_compile" 2>out/conftest.err) ac_status=$? cat out/conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 if (exit $ac_status) && test -s out/conftest2.$ac_objext then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings $ECHO "$_lt_compiler_boilerplate" | $SED '/^$/d' > out/conftest.exp $SED '/^$/d; /^ *+/d' out/conftest.err >out/conftest.er2 if test ! -s out/conftest.er2 || diff out/conftest.exp out/conftest.er2 >/dev/null; then lt_cv_prog_compiler_c_o=yes fi fi chmod u+w . 2>&5 $RM conftest* # SGI C++ compiler will create directory out/ii_files/ for # template instantiation test -d out/ii_files && $RM out/ii_files/* && rmdir out/ii_files $RM out/* && rmdir out cd .. $RM -r conftest $RM conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_c_o" >&5 $as_echo "$lt_cv_prog_compiler_c_o" >&6; } hard_links=nottested if test no = "$lt_cv_prog_compiler_c_o" && test no != "$need_locks"; then # do not overwrite the value of need_locks provided by the user { $as_echo "$as_me:${as_lineno-$LINENO}: checking if we can lock with hard links" >&5 $as_echo_n "checking if we can lock with hard links... " >&6; } hard_links=yes $RM conftest* ln conftest.a conftest.b 2>/dev/null && hard_links=no touch conftest.a ln conftest.a conftest.b 2>&5 || hard_links=no ln conftest.a conftest.b 2>/dev/null && hard_links=no { $as_echo "$as_me:${as_lineno-$LINENO}: result: $hard_links" >&5 $as_echo "$hard_links" >&6; } if test no = "$hard_links"; then { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: '$CC' does not support '-c -o', so 'make -j' may be unsafe" >&5 $as_echo "$as_me: WARNING: '$CC' does not support '-c -o', so 'make -j' may be unsafe" >&2;} need_locks=warn fi else need_locks=no fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether the $compiler linker ($LD) supports shared libraries" >&5 $as_echo_n "checking whether the $compiler linker ($LD) supports shared libraries... " >&6; } runpath_var= allow_undefined_flag= always_export_symbols=no archive_cmds= archive_expsym_cmds= compiler_needs_object=no enable_shared_with_static_runtimes=no export_dynamic_flag_spec= export_symbols_cmds='$NM $libobjs $convenience | $global_symbol_pipe | $SED '\''s/.* //'\'' | sort | uniq > $export_symbols' hardcode_automatic=no hardcode_direct=no hardcode_direct_absolute=no hardcode_libdir_flag_spec= hardcode_libdir_separator= hardcode_minus_L=no hardcode_shlibpath_var=unsupported inherit_rpath=no link_all_deplibs=unknown module_cmds= module_expsym_cmds= old_archive_from_new_cmds= old_archive_from_expsyms_cmds= thread_safe_flag_spec= whole_archive_flag_spec= # include_expsyms should be a list of space-separated symbols to be *always* # included in the symbol list include_expsyms= # exclude_expsyms can be an extended regexp of symbols to exclude # it will be wrapped by ' (' and ')$', so one must not match beginning or # end of line. Example: 'a|bc|.*d.*' will exclude the symbols 'a' and 'bc', # as well as any symbol that contains 'd'. exclude_expsyms='_GLOBAL_OFFSET_TABLE_|_GLOBAL__F[ID]_.*' # Although _GLOBAL_OFFSET_TABLE_ is a valid symbol C name, most a.out # platforms (ab)use it in PIC code, but their linkers get confused if # the symbol is explicitly referenced. Since portable code cannot # rely on this symbol name, it's probably fine to never include it in # preloaded symbol tables. # Exclude shared library initialization/finalization symbols. extract_expsyms_cmds= case $host_os in cygwin* | mingw* | pw32* | cegcc*) # FIXME: the MSVC++ port hasn't been tested in a loooong time # When not using gcc, we currently assume that we are using # Microsoft Visual C++. if test yes != "$GCC"; then with_gnu_ld=no fi ;; interix*) # we just hope/assume this is gcc and not c89 (= MSVC++) with_gnu_ld=yes ;; openbsd* | bitrig*) with_gnu_ld=no ;; linux* | k*bsd*-gnu | gnu*) link_all_deplibs=no ;; esac ld_shlibs=yes # On some targets, GNU ld is compatible enough with the native linker # that we're better off using the native interface for both. lt_use_gnu_ld_interface=no if test yes = "$with_gnu_ld"; then case $host_os in aix*) # The AIX port of GNU ld has always aspired to compatibility # with the native linker. However, as the warning in the GNU ld # block says, versions before 2.19.5* couldn't really create working # shared libraries, regardless of the interface used. case `$LD -v 2>&1` in *\ \(GNU\ Binutils\)\ 2.19.5*) ;; *\ \(GNU\ Binutils\)\ 2.[2-9]*) ;; *\ \(GNU\ Binutils\)\ [3-9]*) ;; *) lt_use_gnu_ld_interface=yes ;; esac ;; *) lt_use_gnu_ld_interface=yes ;; esac fi if test yes = "$lt_use_gnu_ld_interface"; then # If archive_cmds runs LD, not CC, wlarc should be empty wlarc='$wl' # Set some defaults for GNU ld with shared library support. These # are reset later if shared libraries are not supported. Putting them # here allows them to be overridden if necessary. runpath_var=LD_RUN_PATH hardcode_libdir_flag_spec='$wl-rpath $wl$libdir' export_dynamic_flag_spec='$wl--export-dynamic' # ancient GNU ld didn't support --whole-archive et. al. if $LD --help 2>&1 | $GREP 'no-whole-archive' > /dev/null; then whole_archive_flag_spec=$wlarc'--whole-archive$convenience '$wlarc'--no-whole-archive' else whole_archive_flag_spec= fi supports_anon_versioning=no case `$LD -v | $SED -e 's/(^)\+)\s\+//' 2>&1` in *GNU\ gold*) supports_anon_versioning=yes ;; *\ [01].* | *\ 2.[0-9].* | *\ 2.10.*) ;; # catch versions < 2.11 *\ 2.11.93.0.2\ *) supports_anon_versioning=yes ;; # RH7.3 ... *\ 2.11.92.0.12\ *) supports_anon_versioning=yes ;; # Mandrake 8.2 ... *\ 2.11.*) ;; # other 2.11 versions *) supports_anon_versioning=yes ;; esac # See if GNU ld supports shared libraries. case $host_os in aix[3-9]*) # On AIX/PPC, the GNU linker is very broken if test ia64 != "$host_cpu"; then ld_shlibs=no cat <<_LT_EOF 1>&2 *** Warning: the GNU linker, at least up to release 2.19, is reported *** to be unable to reliably create shared libraries on AIX. *** Therefore, libtool is disabling shared libraries support. If you *** really care for shared libraries, you may want to install binutils *** 2.20 or above, or modify your PATH so that a non-GNU linker is found. *** You will then need to restart the configuration process. _LT_EOF fi ;; amigaos*) case $host_cpu in powerpc) # see comment about AmigaOS4 .so support archive_cmds='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' archive_expsym_cmds='' ;; m68k) archive_cmds='$RM $output_objdir/a2ixlibrary.data~$ECHO "#define NAME $libname" > $output_objdir/a2ixlibrary.data~$ECHO "#define LIBRARY_ID 1" >> $output_objdir/a2ixlibrary.data~$ECHO "#define VERSION $major" >> $output_objdir/a2ixlibrary.data~$ECHO "#define REVISION $revision" >> $output_objdir/a2ixlibrary.data~$AR $AR_FLAGS $lib $libobjs~$RANLIB $lib~(cd $output_objdir && a2ixlibrary -32)' hardcode_libdir_flag_spec='-L$libdir' hardcode_minus_L=yes ;; esac ;; beos*) if $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then allow_undefined_flag=unsupported # Joseph Beckenbach says some releases of gcc # support --undefined. This deserves some investigation. FIXME archive_cmds='$CC -nostart $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' else ld_shlibs=no fi ;; cygwin* | mingw* | pw32* | cegcc*) # _LT_TAGVAR(hardcode_libdir_flag_spec, ) is actually meaningless, # as there is no search path for DLLs. hardcode_libdir_flag_spec='-L$libdir' export_dynamic_flag_spec='$wl--export-all-symbols' allow_undefined_flag=unsupported always_export_symbols=no enable_shared_with_static_runtimes=yes export_symbols_cmds='$NM $libobjs $convenience | $global_symbol_pipe | $SED -e '\''/^[BCDGRS][ ]/s/.*[ ]\([^ ]*\)/\1 DATA/;s/^.*[ ]__nm__\([^ ]*\)[ ][^ ]*/\1 DATA/;/^I[ ]/d;/^[AITW][ ]/s/.* //'\'' | sort | uniq > $export_symbols' exclude_expsyms='[_]+GLOBAL_OFFSET_TABLE_|[_]+GLOBAL__[FID]_.*|[_]+head_[A-Za-z0-9_]+_dll|[A-Za-z0-9_]+_dll_iname' if $LD --help 2>&1 | $GREP 'auto-import' > /dev/null; then archive_cmds='$CC -shared $libobjs $deplibs $compiler_flags -o $output_objdir/$soname $wl--enable-auto-image-base -Xlinker --out-implib -Xlinker $lib' # If the export-symbols file already is a .def file, use it as # is; otherwise, prepend EXPORTS... archive_expsym_cmds='if test DEF = "`$SED -n -e '\''s/^[ ]*//'\'' -e '\''/^\(;.*\)*$/d'\'' -e '\''s/^\(EXPORTS\|LIBRARY\)\([ ].*\)*$/DEF/p'\'' -e q $export_symbols`" ; then cp $export_symbols $output_objdir/$soname.def; else echo EXPORTS > $output_objdir/$soname.def; cat $export_symbols >> $output_objdir/$soname.def; fi~ $CC -shared $output_objdir/$soname.def $libobjs $deplibs $compiler_flags -o $output_objdir/$soname $wl--enable-auto-image-base -Xlinker --out-implib -Xlinker $lib' else ld_shlibs=no fi ;; haiku*) archive_cmds='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' link_all_deplibs=yes ;; os2*) hardcode_libdir_flag_spec='-L$libdir' hardcode_minus_L=yes allow_undefined_flag=unsupported shrext_cmds=.dll archive_cmds='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ $ECHO EXPORTS >> $output_objdir/$libname.def~ emxexp $libobjs | $SED /"_DLL_InitTerm"/d >> $output_objdir/$libname.def~ $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ emximp -o $lib $output_objdir/$libname.def' archive_expsym_cmds='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ $ECHO EXPORTS >> $output_objdir/$libname.def~ prefix_cmds="$SED"~ if test EXPORTS = "`$SED 1q $export_symbols`"; then prefix_cmds="$prefix_cmds -e 1d"; fi~ prefix_cmds="$prefix_cmds -e \"s/^\(.*\)$/_\1/g\""~ cat $export_symbols | $prefix_cmds >> $output_objdir/$libname.def~ $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ emximp -o $lib $output_objdir/$libname.def' old_archive_From_new_cmds='emximp -o $output_objdir/${libname}_dll.a $output_objdir/$libname.def' enable_shared_with_static_runtimes=yes ;; interix[3-9]*) hardcode_direct=no hardcode_shlibpath_var=no hardcode_libdir_flag_spec='$wl-rpath,$libdir' export_dynamic_flag_spec='$wl-E' # Hack: On Interix 3.x, we cannot compile PIC because of a broken gcc. # Instead, shared libraries are loaded at an image base (0x10000000 by # default) and relocated if they conflict, which is a slow very memory # consuming and fragmenting process. To avoid this, we pick a random, # 256 KiB-aligned image base between 0x50000000 and 0x6FFC0000 at link # time. Moving up from 0x10000000 also allows more sbrk(2) space. archive_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-h,$soname $wl--image-base,`expr ${RANDOM-$$} % 4096 / 2 \* 262144 + 1342177280` -o $lib' archive_expsym_cmds='sed "s|^|_|" $export_symbols >$output_objdir/$soname.expsym~$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-h,$soname $wl--retain-symbols-file,$output_objdir/$soname.expsym $wl--image-base,`expr ${RANDOM-$$} % 4096 / 2 \* 262144 + 1342177280` -o $lib' ;; gnu* | linux* | tpf* | k*bsd*-gnu | kopensolaris*-gnu) tmp_diet=no if test linux-dietlibc = "$host_os"; then case $cc_basename in diet\ *) tmp_diet=yes;; # linux-dietlibc with static linking (!diet-dyn) esac fi if $LD --help 2>&1 | $EGREP ': supported targets:.* elf' > /dev/null \ && test no = "$tmp_diet" then tmp_addflag=' $pic_flag' tmp_sharedflag='-shared' case $cc_basename,$host_cpu in pgcc*) # Portland Group C compiler whole_archive_flag_spec='$wl--whole-archive`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' tmp_addflag=' $pic_flag' ;; pgf77* | pgf90* | pgf95* | pgfortran*) # Portland Group f77 and f90 compilers whole_archive_flag_spec='$wl--whole-archive`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' tmp_addflag=' $pic_flag -Mnomain' ;; ecc*,ia64* | icc*,ia64*) # Intel C compiler on ia64 tmp_addflag=' -i_dynamic' ;; efc*,ia64* | ifort*,ia64*) # Intel Fortran compiler on ia64 tmp_addflag=' -i_dynamic -nofor_main' ;; ifc* | ifort*) # Intel Fortran compiler tmp_addflag=' -nofor_main' ;; lf95*) # Lahey Fortran 8.1 whole_archive_flag_spec= tmp_sharedflag='--shared' ;; nagfor*) # NAGFOR 5.3 tmp_sharedflag='-Wl,-shared' ;; xl[cC]* | bgxl[cC]* | mpixl[cC]*) # IBM XL C 8.0 on PPC (deal with xlf below) tmp_sharedflag='-qmkshrobj' tmp_addflag= ;; nvcc*) # Cuda Compiler Driver 2.2 whole_archive_flag_spec='$wl--whole-archive`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' compiler_needs_object=yes ;; esac case `$CC -V 2>&1 | sed 5q` in *Sun\ C*) # Sun C 5.9 whole_archive_flag_spec='$wl--whole-archive`new_convenience=; for conv in $convenience\"\"; do test -z \"$conv\" || new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' compiler_needs_object=yes tmp_sharedflag='-G' ;; *Sun\ F*) # Sun Fortran 8.3 tmp_sharedflag='-G' ;; esac archive_cmds='$CC '"$tmp_sharedflag""$tmp_addflag"' $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' if test yes = "$supports_anon_versioning"; then archive_expsym_cmds='echo "{ global:" > $output_objdir/$libname.ver~ cat $export_symbols | sed -e "s/\(.*\)/\1;/" >> $output_objdir/$libname.ver~ echo "local: *; };" >> $output_objdir/$libname.ver~ $CC '"$tmp_sharedflag""$tmp_addflag"' $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-version-script $wl$output_objdir/$libname.ver -o $lib' fi case $cc_basename in tcc*) export_dynamic_flag_spec='-rdynamic' ;; xlf* | bgf* | bgxlf* | mpixlf*) # IBM XL Fortran 10.1 on PPC cannot create shared libs itself whole_archive_flag_spec='--whole-archive$convenience --no-whole-archive' hardcode_libdir_flag_spec='$wl-rpath $wl$libdir' archive_cmds='$LD -shared $libobjs $deplibs $linker_flags -soname $soname -o $lib' if test yes = "$supports_anon_versioning"; then archive_expsym_cmds='echo "{ global:" > $output_objdir/$libname.ver~ cat $export_symbols | sed -e "s/\(.*\)/\1;/" >> $output_objdir/$libname.ver~ echo "local: *; };" >> $output_objdir/$libname.ver~ $LD -shared $libobjs $deplibs $linker_flags -soname $soname -version-script $output_objdir/$libname.ver -o $lib' fi ;; esac else ld_shlibs=no fi ;; netbsd* | netbsdelf*-gnu) if echo __ELF__ | $CC -E - | $GREP __ELF__ >/dev/null; then archive_cmds='$LD -Bshareable $libobjs $deplibs $linker_flags -o $lib' wlarc= else archive_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' archive_expsym_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' fi ;; solaris*) if $LD -v 2>&1 | $GREP 'BFD 2\.8' > /dev/null; then ld_shlibs=no cat <<_LT_EOF 1>&2 *** Warning: The releases 2.8.* of the GNU linker cannot reliably *** create shared libraries on Solaris systems. Therefore, libtool *** is disabling shared libraries support. We urge you to upgrade GNU *** binutils to release 2.9.1 or newer. Another option is to modify *** your PATH or compiler configuration so that the native linker is *** used, and then restart. _LT_EOF elif $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then archive_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' archive_expsym_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' else ld_shlibs=no fi ;; sysv5* | sco3.2v5* | sco5v6* | unixware* | OpenUNIX*) case `$LD -v 2>&1` in *\ [01].* | *\ 2.[0-9].* | *\ 2.1[0-5].*) ld_shlibs=no cat <<_LT_EOF 1>&2 *** Warning: Releases of the GNU linker prior to 2.16.91.0.3 cannot *** reliably create shared libraries on SCO systems. Therefore, libtool *** is disabling shared libraries support. We urge you to upgrade GNU *** binutils to release 2.16.91.0.3 or newer. Another option is to modify *** your PATH or compiler configuration so that the native linker is *** used, and then restart. _LT_EOF ;; *) # For security reasons, it is highly recommended that you always # use absolute paths for naming shared libraries, and exclude the # DT_RUNPATH tag from executables and libraries. But doing so # requires that you compile everything twice, which is a pain. if $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then hardcode_libdir_flag_spec='$wl-rpath $wl$libdir' archive_cmds='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' archive_expsym_cmds='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' else ld_shlibs=no fi ;; esac ;; sunos4*) archive_cmds='$LD -assert pure-text -Bshareable -o $lib $libobjs $deplibs $linker_flags' wlarc= hardcode_direct=yes hardcode_shlibpath_var=no ;; *) if $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then archive_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' archive_expsym_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' else ld_shlibs=no fi ;; esac if test no = "$ld_shlibs"; then runpath_var= hardcode_libdir_flag_spec= export_dynamic_flag_spec= whole_archive_flag_spec= fi else # PORTME fill in a description of your system's linker (not GNU ld) case $host_os in aix3*) allow_undefined_flag=unsupported always_export_symbols=yes archive_expsym_cmds='$LD -o $output_objdir/$soname $libobjs $deplibs $linker_flags -bE:$export_symbols -T512 -H512 -bM:SRE~$AR $AR_FLAGS $lib $output_objdir/$soname' # Note: this linker hardcodes the directories in LIBPATH if there # are no directories specified by -L. hardcode_minus_L=yes if test yes = "$GCC" && test -z "$lt_prog_compiler_static"; then # Neither direct hardcoding nor static linking is supported with a # broken collect2. hardcode_direct=unsupported fi ;; aix[4-9]*) if test ia64 = "$host_cpu"; then # On IA64, the linker does run time linking by default, so we don't # have to do anything special. aix_use_runtimelinking=no exp_sym_flag='-Bexport' no_entry_flag= else # If we're using GNU nm, then we don't want the "-C" option. # -C means demangle to GNU nm, but means don't demangle to AIX nm. # Without the "-l" option, or with the "-B" option, AIX nm treats # weak defined symbols like other global defined symbols, whereas # GNU nm marks them as "W". # While the 'weak' keyword is ignored in the Export File, we need # it in the Import File for the 'aix-soname' feature, so we have # to replace the "-B" option with "-P" for AIX nm. if $NM -V 2>&1 | $GREP 'GNU' > /dev/null; then export_symbols_cmds='$NM -Bpg $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W")) && (substr(\$ 3,1,1) != ".")) { if (\$ 2 == "W") { print \$ 3 " weak" } else { print \$ 3 } } }'\'' | sort -u > $export_symbols' else export_symbols_cmds='`func_echo_all $NM | $SED -e '\''s/B\([^B]*\)$/P\1/'\''` -PCpgl $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) && (substr(\$ 1,1,1) != ".")) { if ((\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) { print \$ 1 " weak" } else { print \$ 1 } } }'\'' | sort -u > $export_symbols' fi aix_use_runtimelinking=no # Test if we are trying to use run time linking or normal # AIX style linking. If -brtl is somewhere in LDFLAGS, we # have runtime linking enabled, and use it for executables. # For shared libraries, we enable/disable runtime linking # depending on the kind of the shared library created - # when "with_aix_soname,aix_use_runtimelinking" is: # "aix,no" lib.a(lib.so.V) shared, rtl:no, for executables # "aix,yes" lib.so shared, rtl:yes, for executables # lib.a static archive # "both,no" lib.so.V(shr.o) shared, rtl:yes # lib.a(lib.so.V) shared, rtl:no, for executables # "both,yes" lib.so.V(shr.o) shared, rtl:yes, for executables # lib.a(lib.so.V) shared, rtl:no # "svr4,*" lib.so.V(shr.o) shared, rtl:yes, for executables # lib.a static archive case $host_os in aix4.[23]|aix4.[23].*|aix[5-9]*) for ld_flag in $LDFLAGS; do if (test x-brtl = "x$ld_flag" || test x-Wl,-brtl = "x$ld_flag"); then aix_use_runtimelinking=yes break fi done if test svr4,no = "$with_aix_soname,$aix_use_runtimelinking"; then # With aix-soname=svr4, we create the lib.so.V shared archives only, # so we don't have lib.a shared libs to link our executables. # We have to force runtime linking in this case. aix_use_runtimelinking=yes LDFLAGS="$LDFLAGS -Wl,-brtl" fi ;; esac exp_sym_flag='-bexport' no_entry_flag='-bnoentry' fi # When large executables or shared objects are built, AIX ld can # have problems creating the table of contents. If linking a library # or program results in "error TOC overflow" add -mminimal-toc to # CXXFLAGS/CFLAGS for g++/gcc. In the cases where that is not # enough to fix the problem, add -Wl,-bbigtoc to LDFLAGS. archive_cmds='' hardcode_direct=yes hardcode_direct_absolute=yes hardcode_libdir_separator=':' link_all_deplibs=yes file_list_spec='$wl-f,' case $with_aix_soname,$aix_use_runtimelinking in aix,*) ;; # traditional, no import file svr4,* | *,yes) # use import file # The Import File defines what to hardcode. hardcode_direct=no hardcode_direct_absolute=no ;; esac if test yes = "$GCC"; then case $host_os in aix4.[012]|aix4.[012].*) # We only want to do this on AIX 4.2 and lower, the check # below for broken collect2 doesn't work under 4.3+ collect2name=`$CC -print-prog-name=collect2` if test -f "$collect2name" && strings "$collect2name" | $GREP resolve_lib_name >/dev/null then # We have reworked collect2 : else # We have old collect2 hardcode_direct=unsupported # It fails to find uninstalled libraries when the uninstalled # path is not listed in the libpath. Setting hardcode_minus_L # to unsupported forces relinking hardcode_minus_L=yes hardcode_libdir_flag_spec='-L$libdir' hardcode_libdir_separator= fi ;; esac shared_flag='-shared' if test yes = "$aix_use_runtimelinking"; then shared_flag="$shared_flag "'$wl-G' fi # Need to ensure runtime linking is disabled for the traditional # shared library, or the linker may eventually find shared libraries # /with/ Import File - we do not want to mix them. shared_flag_aix='-shared' shared_flag_svr4='-shared $wl-G' else # not using gcc if test ia64 = "$host_cpu"; then # VisualAge C++, Version 5.5 for AIX 5L for IA-64, Beta 3 Release # chokes on -Wl,-G. The following line is correct: shared_flag='-G' else if test yes = "$aix_use_runtimelinking"; then shared_flag='$wl-G' else shared_flag='$wl-bM:SRE' fi shared_flag_aix='$wl-bM:SRE' shared_flag_svr4='$wl-G' fi fi export_dynamic_flag_spec='$wl-bexpall' # It seems that -bexpall does not export symbols beginning with # underscore (_), so it is better to generate a list of symbols to export. always_export_symbols=yes if test aix,yes = "$with_aix_soname,$aix_use_runtimelinking"; then # Warning - without using the other runtime loading flags (-brtl), # -berok will link without error, but may produce a broken library. allow_undefined_flag='-berok' # Determine the default libpath from the value encoded in an # empty executable. if test set = "${lt_cv_aix_libpath+set}"; then aix_libpath=$lt_cv_aix_libpath else if ${lt_cv_aix_libpath_+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : lt_aix_libpath_sed=' /Import File Strings/,/^$/ { /^0/ { s/^0 *\([^ ]*\) *$/\1/ p } }' lt_cv_aix_libpath_=`dump -H conftest$ac_exeext 2>/dev/null | $SED -n -e "$lt_aix_libpath_sed"` # Check for a 64-bit object if we didn't find anything. if test -z "$lt_cv_aix_libpath_"; then lt_cv_aix_libpath_=`dump -HX64 conftest$ac_exeext 2>/dev/null | $SED -n -e "$lt_aix_libpath_sed"` fi fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext if test -z "$lt_cv_aix_libpath_"; then lt_cv_aix_libpath_=/usr/lib:/lib fi fi aix_libpath=$lt_cv_aix_libpath_ fi hardcode_libdir_flag_spec='$wl-blibpath:$libdir:'"$aix_libpath" archive_expsym_cmds='$CC -o $output_objdir/$soname $libobjs $deplibs $wl'$no_entry_flag' $compiler_flags `if test -n "$allow_undefined_flag"; then func_echo_all "$wl$allow_undefined_flag"; else :; fi` $wl'$exp_sym_flag:\$export_symbols' '$shared_flag else if test ia64 = "$host_cpu"; then hardcode_libdir_flag_spec='$wl-R $libdir:/usr/lib:/lib' allow_undefined_flag="-z nodefs" archive_expsym_cmds="\$CC $shared_flag"' -o $output_objdir/$soname $libobjs $deplibs '"\$wl$no_entry_flag"' $compiler_flags $wl$allow_undefined_flag '"\$wl$exp_sym_flag:\$export_symbols" else # Determine the default libpath from the value encoded in an # empty executable. if test set = "${lt_cv_aix_libpath+set}"; then aix_libpath=$lt_cv_aix_libpath else if ${lt_cv_aix_libpath_+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : lt_aix_libpath_sed=' /Import File Strings/,/^$/ { /^0/ { s/^0 *\([^ ]*\) *$/\1/ p } }' lt_cv_aix_libpath_=`dump -H conftest$ac_exeext 2>/dev/null | $SED -n -e "$lt_aix_libpath_sed"` # Check for a 64-bit object if we didn't find anything. if test -z "$lt_cv_aix_libpath_"; then lt_cv_aix_libpath_=`dump -HX64 conftest$ac_exeext 2>/dev/null | $SED -n -e "$lt_aix_libpath_sed"` fi fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext if test -z "$lt_cv_aix_libpath_"; then lt_cv_aix_libpath_=/usr/lib:/lib fi fi aix_libpath=$lt_cv_aix_libpath_ fi hardcode_libdir_flag_spec='$wl-blibpath:$libdir:'"$aix_libpath" # Warning - without using the other run time loading flags, # -berok will link without error, but may produce a broken library. no_undefined_flag=' $wl-bernotok' allow_undefined_flag=' $wl-berok' if test yes = "$with_gnu_ld"; then # We only use this code for GNU lds that support --whole-archive. whole_archive_flag_spec='$wl--whole-archive$convenience $wl--no-whole-archive' else # Exported symbols can be pulled into shared objects from archives whole_archive_flag_spec='$convenience' fi archive_cmds_need_lc=yes archive_expsym_cmds='$RM -r $output_objdir/$realname.d~$MKDIR $output_objdir/$realname.d' # -brtl affects multiple linker settings, -berok does not and is overridden later compiler_flags_filtered='`func_echo_all "$compiler_flags " | $SED -e "s%-brtl\\([, ]\\)%-berok\\1%g"`' if test svr4 != "$with_aix_soname"; then # This is similar to how AIX traditionally builds its shared libraries. archive_expsym_cmds="$archive_expsym_cmds"'~$CC '$shared_flag_aix' -o $output_objdir/$realname.d/$soname $libobjs $deplibs $wl-bnoentry '$compiler_flags_filtered'$wl-bE:$export_symbols$allow_undefined_flag~$AR $AR_FLAGS $output_objdir/$libname$release.a $output_objdir/$realname.d/$soname' fi if test aix != "$with_aix_soname"; then archive_expsym_cmds="$archive_expsym_cmds"'~$CC '$shared_flag_svr4' -o $output_objdir/$realname.d/$shared_archive_member_spec.o $libobjs $deplibs $wl-bnoentry '$compiler_flags_filtered'$wl-bE:$export_symbols$allow_undefined_flag~$STRIP -e $output_objdir/$realname.d/$shared_archive_member_spec.o~( func_echo_all "#! $soname($shared_archive_member_spec.o)"; if test shr_64 = "$shared_archive_member_spec"; then func_echo_all "# 64"; else func_echo_all "# 32"; fi; cat $export_symbols ) > $output_objdir/$realname.d/$shared_archive_member_spec.imp~$AR $AR_FLAGS $output_objdir/$soname $output_objdir/$realname.d/$shared_archive_member_spec.o $output_objdir/$realname.d/$shared_archive_member_spec.imp' else # used by -dlpreopen to get the symbols archive_expsym_cmds="$archive_expsym_cmds"'~$MV $output_objdir/$realname.d/$soname $output_objdir' fi archive_expsym_cmds="$archive_expsym_cmds"'~$RM -r $output_objdir/$realname.d' fi fi ;; amigaos*) case $host_cpu in powerpc) # see comment about AmigaOS4 .so support archive_cmds='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' archive_expsym_cmds='' ;; m68k) archive_cmds='$RM $output_objdir/a2ixlibrary.data~$ECHO "#define NAME $libname" > $output_objdir/a2ixlibrary.data~$ECHO "#define LIBRARY_ID 1" >> $output_objdir/a2ixlibrary.data~$ECHO "#define VERSION $major" >> $output_objdir/a2ixlibrary.data~$ECHO "#define REVISION $revision" >> $output_objdir/a2ixlibrary.data~$AR $AR_FLAGS $lib $libobjs~$RANLIB $lib~(cd $output_objdir && a2ixlibrary -32)' hardcode_libdir_flag_spec='-L$libdir' hardcode_minus_L=yes ;; esac ;; bsdi[45]*) export_dynamic_flag_spec=-rdynamic ;; cygwin* | mingw* | pw32* | cegcc*) # When not using gcc, we currently assume that we are using # Microsoft Visual C++. # hardcode_libdir_flag_spec is actually meaningless, as there is # no search path for DLLs. case $cc_basename in cl*) # Native MSVC hardcode_libdir_flag_spec=' ' allow_undefined_flag=unsupported always_export_symbols=yes file_list_spec='@' # Tell ltmain to make .lib files, not .a files. libext=lib # Tell ltmain to make .dll files, not .so files. shrext_cmds=.dll # FIXME: Setting linknames here is a bad hack. archive_cmds='$CC -o $output_objdir/$soname $libobjs $compiler_flags $deplibs -Wl,-DLL,-IMPLIB:"$tool_output_objdir$libname.dll.lib"~linknames=' archive_expsym_cmds='if test DEF = "`$SED -n -e '\''s/^[ ]*//'\'' -e '\''/^\(;.*\)*$/d'\'' -e '\''s/^\(EXPORTS\|LIBRARY\)\([ ].*\)*$/DEF/p'\'' -e q $export_symbols`" ; then cp "$export_symbols" "$output_objdir/$soname.def"; echo "$tool_output_objdir$soname.def" > "$output_objdir/$soname.exp"; else $SED -e '\''s/^/-link -EXPORT:/'\'' < $export_symbols > $output_objdir/$soname.exp; fi~ $CC -o $tool_output_objdir$soname $libobjs $compiler_flags $deplibs "@$tool_output_objdir$soname.exp" -Wl,-DLL,-IMPLIB:"$tool_output_objdir$libname.dll.lib"~ linknames=' # The linker will not automatically build a static lib if we build a DLL. # _LT_TAGVAR(old_archive_from_new_cmds, )='true' enable_shared_with_static_runtimes=yes exclude_expsyms='_NULL_IMPORT_DESCRIPTOR|_IMPORT_DESCRIPTOR_.*' export_symbols_cmds='$NM $libobjs $convenience | $global_symbol_pipe | $SED -e '\''/^[BCDGRS][ ]/s/.*[ ]\([^ ]*\)/\1,DATA/'\'' | $SED -e '\''/^[AITW][ ]/s/.*[ ]//'\'' | sort | uniq > $export_symbols' # Don't use ranlib old_postinstall_cmds='chmod 644 $oldlib' postlink_cmds='lt_outputfile="@OUTPUT@"~ lt_tool_outputfile="@TOOL_OUTPUT@"~ case $lt_outputfile in *.exe|*.EXE) ;; *) lt_outputfile=$lt_outputfile.exe lt_tool_outputfile=$lt_tool_outputfile.exe ;; esac~ if test : != "$MANIFEST_TOOL" && test -f "$lt_outputfile.manifest"; then $MANIFEST_TOOL -manifest "$lt_tool_outputfile.manifest" -outputresource:"$lt_tool_outputfile" || exit 1; $RM "$lt_outputfile.manifest"; fi' ;; *) # Assume MSVC wrapper hardcode_libdir_flag_spec=' ' allow_undefined_flag=unsupported # Tell ltmain to make .lib files, not .a files. libext=lib # Tell ltmain to make .dll files, not .so files. shrext_cmds=.dll # FIXME: Setting linknames here is a bad hack. archive_cmds='$CC -o $lib $libobjs $compiler_flags `func_echo_all "$deplibs" | $SED '\''s/ -lc$//'\''` -link -dll~linknames=' # The linker will automatically build a .lib file if we build a DLL. old_archive_from_new_cmds='true' # FIXME: Should let the user specify the lib program. old_archive_cmds='lib -OUT:$oldlib$oldobjs$old_deplibs' enable_shared_with_static_runtimes=yes ;; esac ;; darwin* | rhapsody*) archive_cmds_need_lc=no hardcode_direct=no hardcode_automatic=yes hardcode_shlibpath_var=unsupported if test yes = "$lt_cv_ld_force_load"; then whole_archive_flag_spec='`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience $wl-force_load,$conv\"; done; func_echo_all \"$new_convenience\"`' else whole_archive_flag_spec='' fi link_all_deplibs=yes allow_undefined_flag=$_lt_dar_allow_undefined case $cc_basename in ifort*|nagfor*) _lt_dar_can_shared=yes ;; *) _lt_dar_can_shared=$GCC ;; esac if test yes = "$_lt_dar_can_shared"; then output_verbose_link_cmd=func_echo_all archive_cmds="\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$libobjs \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring $_lt_dar_single_mod$_lt_dsymutil" module_cmds="\$CC \$allow_undefined_flag -o \$lib -bundle \$libobjs \$deplibs \$compiler_flags$_lt_dsymutil" archive_expsym_cmds="sed 's|^|_|' < \$export_symbols > \$output_objdir/\$libname-symbols.expsym~\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$libobjs \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring $_lt_dar_single_mod$_lt_dar_export_syms$_lt_dsymutil" module_expsym_cmds="sed -e 's|^|_|' < \$export_symbols > \$output_objdir/\$libname-symbols.expsym~\$CC \$allow_undefined_flag -o \$lib -bundle \$libobjs \$deplibs \$compiler_flags$_lt_dar_export_syms$_lt_dsymutil" else ld_shlibs=no fi ;; dgux*) archive_cmds='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' hardcode_libdir_flag_spec='-L$libdir' hardcode_shlibpath_var=no ;; # FreeBSD 2.2.[012] allows us to include c++rt0.o to get C++ constructor # support. Future versions do this automatically, but an explicit c++rt0.o # does not break anything, and helps significantly (at the cost of a little # extra space). freebsd2.2*) archive_cmds='$LD -Bshareable -o $lib $libobjs $deplibs $linker_flags /usr/lib/c++rt0.o' hardcode_libdir_flag_spec='-R$libdir' hardcode_direct=yes hardcode_shlibpath_var=no ;; # Unfortunately, older versions of FreeBSD 2 do not have this feature. freebsd2.*) archive_cmds='$LD -Bshareable -o $lib $libobjs $deplibs $linker_flags' hardcode_direct=yes hardcode_minus_L=yes hardcode_shlibpath_var=no ;; # FreeBSD 3 and greater uses gcc -shared to do shared libraries. freebsd* | dragonfly*) archive_cmds='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags' hardcode_libdir_flag_spec='-R$libdir' hardcode_direct=yes hardcode_shlibpath_var=no ;; hpux9*) if test yes = "$GCC"; then archive_cmds='$RM $output_objdir/$soname~$CC -shared $pic_flag $wl+b $wl$install_libdir -o $output_objdir/$soname $libobjs $deplibs $compiler_flags~test "x$output_objdir/$soname" = "x$lib" || mv $output_objdir/$soname $lib' else archive_cmds='$RM $output_objdir/$soname~$LD -b +b $install_libdir -o $output_objdir/$soname $libobjs $deplibs $linker_flags~test "x$output_objdir/$soname" = "x$lib" || mv $output_objdir/$soname $lib' fi hardcode_libdir_flag_spec='$wl+b $wl$libdir' hardcode_libdir_separator=: hardcode_direct=yes # hardcode_minus_L: Not really in the search PATH, # but as the default location of the library. hardcode_minus_L=yes export_dynamic_flag_spec='$wl-E' ;; hpux10*) if test yes,no = "$GCC,$with_gnu_ld"; then archive_cmds='$CC -shared $pic_flag $wl+h $wl$soname $wl+b $wl$install_libdir -o $lib $libobjs $deplibs $compiler_flags' else archive_cmds='$LD -b +h $soname +b $install_libdir -o $lib $libobjs $deplibs $linker_flags' fi if test no = "$with_gnu_ld"; then hardcode_libdir_flag_spec='$wl+b $wl$libdir' hardcode_libdir_separator=: hardcode_direct=yes hardcode_direct_absolute=yes export_dynamic_flag_spec='$wl-E' # hardcode_minus_L: Not really in the search PATH, # but as the default location of the library. hardcode_minus_L=yes fi ;; hpux11*) if test yes,no = "$GCC,$with_gnu_ld"; then case $host_cpu in hppa*64*) archive_cmds='$CC -shared $wl+h $wl$soname -o $lib $libobjs $deplibs $compiler_flags' ;; ia64*) archive_cmds='$CC -shared $pic_flag $wl+h $wl$soname $wl+nodefaultrpath -o $lib $libobjs $deplibs $compiler_flags' ;; *) archive_cmds='$CC -shared $pic_flag $wl+h $wl$soname $wl+b $wl$install_libdir -o $lib $libobjs $deplibs $compiler_flags' ;; esac else case $host_cpu in hppa*64*) archive_cmds='$CC -b $wl+h $wl$soname -o $lib $libobjs $deplibs $compiler_flags' ;; ia64*) archive_cmds='$CC -b $wl+h $wl$soname $wl+nodefaultrpath -o $lib $libobjs $deplibs $compiler_flags' ;; *) # Older versions of the 11.00 compiler do not understand -b yet # (HP92453-01 A.11.01.20 doesn't, HP92453-01 B.11.X.35175-35176.GP does) { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $CC understands -b" >&5 $as_echo_n "checking if $CC understands -b... " >&6; } if ${lt_cv_prog_compiler__b+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_prog_compiler__b=no save_LDFLAGS=$LDFLAGS LDFLAGS="$LDFLAGS -b" echo "$lt_simple_link_test_code" > conftest.$ac_ext if (eval $ac_link 2>conftest.err) && test -s conftest$ac_exeext; then # The linker can only warn and ignore the option if not recognized # So say no if there are warnings if test -s conftest.err; then # Append any errors to the config.log. cat conftest.err 1>&5 $ECHO "$_lt_linker_boilerplate" | $SED '/^$/d' > conftest.exp $SED '/^$/d; /^ *+/d' conftest.err >conftest.er2 if diff conftest.exp conftest.er2 >/dev/null; then lt_cv_prog_compiler__b=yes fi else lt_cv_prog_compiler__b=yes fi fi $RM -r conftest* LDFLAGS=$save_LDFLAGS fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler__b" >&5 $as_echo "$lt_cv_prog_compiler__b" >&6; } if test yes = "$lt_cv_prog_compiler__b"; then archive_cmds='$CC -b $wl+h $wl$soname $wl+b $wl$install_libdir -o $lib $libobjs $deplibs $compiler_flags' else archive_cmds='$LD -b +h $soname +b $install_libdir -o $lib $libobjs $deplibs $linker_flags' fi ;; esac fi if test no = "$with_gnu_ld"; then hardcode_libdir_flag_spec='$wl+b $wl$libdir' hardcode_libdir_separator=: case $host_cpu in hppa*64*|ia64*) hardcode_direct=no hardcode_shlibpath_var=no ;; *) hardcode_direct=yes hardcode_direct_absolute=yes export_dynamic_flag_spec='$wl-E' # hardcode_minus_L: Not really in the search PATH, # but as the default location of the library. hardcode_minus_L=yes ;; esac fi ;; irix5* | irix6* | nonstopux*) if test yes = "$GCC"; then archive_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations -o $lib' # Try to use the -exported_symbol ld option, if it does not # work, assume that -exports_file does not work either and # implicitly export all symbols. # This should be the same for all languages, so no per-tag cache variable. { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether the $host_os linker accepts -exported_symbol" >&5 $as_echo_n "checking whether the $host_os linker accepts -exported_symbol... " >&6; } if ${lt_cv_irix_exported_symbol+:} false; then : $as_echo_n "(cached) " >&6 else save_LDFLAGS=$LDFLAGS LDFLAGS="$LDFLAGS -shared $wl-exported_symbol ${wl}foo $wl-update_registry $wl/dev/null" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int foo (void) { return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : lt_cv_irix_exported_symbol=yes else lt_cv_irix_exported_symbol=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LDFLAGS=$save_LDFLAGS fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_irix_exported_symbol" >&5 $as_echo "$lt_cv_irix_exported_symbol" >&6; } if test yes = "$lt_cv_irix_exported_symbol"; then archive_expsym_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations $wl-exports_file $wl$export_symbols -o $lib' fi link_all_deplibs=no else archive_cmds='$CC -shared $libobjs $deplibs $compiler_flags -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib' archive_expsym_cmds='$CC -shared $libobjs $deplibs $compiler_flags -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -exports_file $export_symbols -o $lib' fi archive_cmds_need_lc='no' hardcode_libdir_flag_spec='$wl-rpath $wl$libdir' hardcode_libdir_separator=: inherit_rpath=yes link_all_deplibs=yes ;; linux*) case $cc_basename in tcc*) # Fabrice Bellard et al's Tiny C Compiler ld_shlibs=yes archive_cmds='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags' ;; esac ;; netbsd* | netbsdelf*-gnu) if echo __ELF__ | $CC -E - | $GREP __ELF__ >/dev/null; then archive_cmds='$LD -Bshareable -o $lib $libobjs $deplibs $linker_flags' # a.out else archive_cmds='$LD -shared -o $lib $libobjs $deplibs $linker_flags' # ELF fi hardcode_libdir_flag_spec='-R$libdir' hardcode_direct=yes hardcode_shlibpath_var=no ;; newsos6) archive_cmds='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' hardcode_direct=yes hardcode_libdir_flag_spec='$wl-rpath $wl$libdir' hardcode_libdir_separator=: hardcode_shlibpath_var=no ;; *nto* | *qnx*) ;; openbsd* | bitrig*) if test -f /usr/libexec/ld.so; then hardcode_direct=yes hardcode_shlibpath_var=no hardcode_direct_absolute=yes if test -z "`echo __ELF__ | $CC -E - | $GREP __ELF__`"; then archive_cmds='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags' archive_expsym_cmds='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags $wl-retain-symbols-file,$export_symbols' hardcode_libdir_flag_spec='$wl-rpath,$libdir' export_dynamic_flag_spec='$wl-E' else archive_cmds='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags' hardcode_libdir_flag_spec='$wl-rpath,$libdir' fi else ld_shlibs=no fi ;; os2*) hardcode_libdir_flag_spec='-L$libdir' hardcode_minus_L=yes allow_undefined_flag=unsupported shrext_cmds=.dll archive_cmds='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ $ECHO EXPORTS >> $output_objdir/$libname.def~ emxexp $libobjs | $SED /"_DLL_InitTerm"/d >> $output_objdir/$libname.def~ $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ emximp -o $lib $output_objdir/$libname.def' archive_expsym_cmds='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ $ECHO EXPORTS >> $output_objdir/$libname.def~ prefix_cmds="$SED"~ if test EXPORTS = "`$SED 1q $export_symbols`"; then prefix_cmds="$prefix_cmds -e 1d"; fi~ prefix_cmds="$prefix_cmds -e \"s/^\(.*\)$/_\1/g\""~ cat $export_symbols | $prefix_cmds >> $output_objdir/$libname.def~ $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ emximp -o $lib $output_objdir/$libname.def' old_archive_From_new_cmds='emximp -o $output_objdir/${libname}_dll.a $output_objdir/$libname.def' enable_shared_with_static_runtimes=yes ;; osf3*) if test yes = "$GCC"; then allow_undefined_flag=' $wl-expect_unresolved $wl\*' archive_cmds='$CC -shared$allow_undefined_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations -o $lib' else allow_undefined_flag=' -expect_unresolved \*' archive_cmds='$CC -shared$allow_undefined_flag $libobjs $deplibs $compiler_flags -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib' fi archive_cmds_need_lc='no' hardcode_libdir_flag_spec='$wl-rpath $wl$libdir' hardcode_libdir_separator=: ;; osf4* | osf5*) # as osf3* with the addition of -msym flag if test yes = "$GCC"; then allow_undefined_flag=' $wl-expect_unresolved $wl\*' archive_cmds='$CC -shared$allow_undefined_flag $pic_flag $libobjs $deplibs $compiler_flags $wl-msym $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations -o $lib' hardcode_libdir_flag_spec='$wl-rpath $wl$libdir' else allow_undefined_flag=' -expect_unresolved \*' archive_cmds='$CC -shared$allow_undefined_flag $libobjs $deplibs $compiler_flags -msym -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib' archive_expsym_cmds='for i in `cat $export_symbols`; do printf "%s %s\\n" -exported_symbol "\$i" >> $lib.exp; done; printf "%s\\n" "-hidden">> $lib.exp~ $CC -shared$allow_undefined_flag $wl-input $wl$lib.exp $compiler_flags $libobjs $deplibs -soname $soname `test -n "$verstring" && $ECHO "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib~$RM $lib.exp' # Both c and cxx compiler support -rpath directly hardcode_libdir_flag_spec='-rpath $libdir' fi archive_cmds_need_lc='no' hardcode_libdir_separator=: ;; solaris*) no_undefined_flag=' -z defs' if test yes = "$GCC"; then wlarc='$wl' archive_cmds='$CC -shared $pic_flag $wl-z ${wl}text $wl-h $wl$soname -o $lib $libobjs $deplibs $compiler_flags' archive_expsym_cmds='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ $CC -shared $pic_flag $wl-z ${wl}text $wl-M $wl$lib.exp $wl-h $wl$soname -o $lib $libobjs $deplibs $compiler_flags~$RM $lib.exp' else case `$CC -V 2>&1` in *"Compilers 5.0"*) wlarc='' archive_cmds='$LD -G$allow_undefined_flag -h $soname -o $lib $libobjs $deplibs $linker_flags' archive_expsym_cmds='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ $LD -G$allow_undefined_flag -M $lib.exp -h $soname -o $lib $libobjs $deplibs $linker_flags~$RM $lib.exp' ;; *) wlarc='$wl' archive_cmds='$CC -G$allow_undefined_flag -h $soname -o $lib $libobjs $deplibs $compiler_flags' archive_expsym_cmds='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ $CC -G$allow_undefined_flag -M $lib.exp -h $soname -o $lib $libobjs $deplibs $compiler_flags~$RM $lib.exp' ;; esac fi hardcode_libdir_flag_spec='-R$libdir' hardcode_shlibpath_var=no case $host_os in solaris2.[0-5] | solaris2.[0-5].*) ;; *) # The compiler driver will combine and reorder linker options, # but understands '-z linker_flag'. GCC discards it without '$wl', # but is careful enough not to reorder. # Supported since Solaris 2.6 (maybe 2.5.1?) if test yes = "$GCC"; then whole_archive_flag_spec='$wl-z ${wl}allextract$convenience $wl-z ${wl}defaultextract' else whole_archive_flag_spec='-z allextract$convenience -z defaultextract' fi ;; esac link_all_deplibs=yes ;; sunos4*) if test sequent = "$host_vendor"; then # Use $CC to link under sequent, because it throws in some extra .o # files that make .init and .fini sections work. archive_cmds='$CC -G $wl-h $soname -o $lib $libobjs $deplibs $compiler_flags' else archive_cmds='$LD -assert pure-text -Bstatic -o $lib $libobjs $deplibs $linker_flags' fi hardcode_libdir_flag_spec='-L$libdir' hardcode_direct=yes hardcode_minus_L=yes hardcode_shlibpath_var=no ;; sysv4) case $host_vendor in sni) archive_cmds='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' hardcode_direct=yes # is this really true??? ;; siemens) ## LD is ld it makes a PLAMLIB ## CC just makes a GrossModule. archive_cmds='$LD -G -o $lib $libobjs $deplibs $linker_flags' reload_cmds='$CC -r -o $output$reload_objs' hardcode_direct=no ;; motorola) archive_cmds='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' hardcode_direct=no #Motorola manual says yes, but my tests say they lie ;; esac runpath_var='LD_RUN_PATH' hardcode_shlibpath_var=no ;; sysv4.3*) archive_cmds='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' hardcode_shlibpath_var=no export_dynamic_flag_spec='-Bexport' ;; sysv4*MP*) if test -d /usr/nec; then archive_cmds='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' hardcode_shlibpath_var=no runpath_var=LD_RUN_PATH hardcode_runpath_var=yes ld_shlibs=yes fi ;; sysv4*uw2* | sysv5OpenUNIX* | sysv5UnixWare7.[01].[10]* | unixware7* | sco3.2v5.0.[024]*) no_undefined_flag='$wl-z,text' archive_cmds_need_lc=no hardcode_shlibpath_var=no runpath_var='LD_RUN_PATH' if test yes = "$GCC"; then archive_cmds='$CC -shared $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' archive_expsym_cmds='$CC -shared $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' else archive_cmds='$CC -G $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' archive_expsym_cmds='$CC -G $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' fi ;; sysv5* | sco3.2v5* | sco5v6*) # Note: We CANNOT use -z defs as we might desire, because we do not # link with -lc, and that would cause any symbols used from libc to # always be unresolved, which means just about no library would # ever link correctly. If we're not using GNU ld we use -z text # though, which does catch some bad symbols but isn't as heavy-handed # as -z defs. no_undefined_flag='$wl-z,text' allow_undefined_flag='$wl-z,nodefs' archive_cmds_need_lc=no hardcode_shlibpath_var=no hardcode_libdir_flag_spec='$wl-R,$libdir' hardcode_libdir_separator=':' link_all_deplibs=yes export_dynamic_flag_spec='$wl-Bexport' runpath_var='LD_RUN_PATH' if test yes = "$GCC"; then archive_cmds='$CC -shared $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' archive_expsym_cmds='$CC -shared $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' else archive_cmds='$CC -G $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' archive_expsym_cmds='$CC -G $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' fi ;; uts4*) archive_cmds='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' hardcode_libdir_flag_spec='-L$libdir' hardcode_shlibpath_var=no ;; *) ld_shlibs=no ;; esac if test sni = "$host_vendor"; then case $host in sysv4 | sysv4.2uw2* | sysv4.3* | sysv5*) export_dynamic_flag_spec='$wl-Blargedynsym' ;; esac fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ld_shlibs" >&5 $as_echo "$ld_shlibs" >&6; } test no = "$ld_shlibs" && can_build_shared=no with_gnu_ld=$with_gnu_ld # # Do we need to explicitly link libc? # case "x$archive_cmds_need_lc" in x|xyes) # Assume -lc should be added archive_cmds_need_lc=yes if test yes,yes = "$GCC,$enable_shared"; then case $archive_cmds in *'~'*) # FIXME: we may have to deal with multi-command sequences. ;; '$CC '*) # Test whether the compiler implicitly links with -lc since on some # systems, -lgcc has to come before -lc. If gcc already passes -lc # to ld, don't add -lc before -lgcc. { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether -lc should be explicitly linked in" >&5 $as_echo_n "checking whether -lc should be explicitly linked in... " >&6; } if ${lt_cv_archive_cmds_need_lc+:} false; then : $as_echo_n "(cached) " >&6 else $RM conftest* echo "$lt_simple_compile_test_code" > conftest.$ac_ext if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 (eval $ac_compile) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } 2>conftest.err; then soname=conftest lib=conftest libobjs=conftest.$ac_objext deplibs= wl=$lt_prog_compiler_wl pic_flag=$lt_prog_compiler_pic compiler_flags=-v linker_flags=-v verstring= output_objdir=. libname=conftest lt_save_allow_undefined_flag=$allow_undefined_flag allow_undefined_flag= if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$archive_cmds 2\>\&1 \| $GREP \" -lc \" \>/dev/null 2\>\&1\""; } >&5 (eval $archive_cmds 2\>\&1 \| $GREP \" -lc \" \>/dev/null 2\>\&1) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } then lt_cv_archive_cmds_need_lc=no else lt_cv_archive_cmds_need_lc=yes fi allow_undefined_flag=$lt_save_allow_undefined_flag else cat conftest.err 1>&5 fi $RM conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_archive_cmds_need_lc" >&5 $as_echo "$lt_cv_archive_cmds_need_lc" >&6; } archive_cmds_need_lc=$lt_cv_archive_cmds_need_lc ;; esac fi ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: checking dynamic linker characteristics" >&5 $as_echo_n "checking dynamic linker characteristics... " >&6; } if test yes = "$GCC"; then case $host_os in darwin*) lt_awk_arg='/^libraries:/,/LR/' ;; *) lt_awk_arg='/^libraries:/' ;; esac case $host_os in mingw* | cegcc*) lt_sed_strip_eq='s|=\([A-Za-z]:\)|\1|g' ;; *) lt_sed_strip_eq='s|=/|/|g' ;; esac lt_search_path_spec=`$CC -print-search-dirs | awk $lt_awk_arg | $SED -e "s/^libraries://" -e $lt_sed_strip_eq` case $lt_search_path_spec in *\;*) # if the path contains ";" then we assume it to be the separator # otherwise default to the standard path separator (i.e. ":") - it is # assumed that no part of a normal pathname contains ";" but that should # okay in the real world where ";" in dirpaths is itself problematic. lt_search_path_spec=`$ECHO "$lt_search_path_spec" | $SED 's/;/ /g'` ;; *) lt_search_path_spec=`$ECHO "$lt_search_path_spec" | $SED "s/$PATH_SEPARATOR/ /g"` ;; esac # Ok, now we have the path, separated by spaces, we can step through it # and add multilib dir if necessary... lt_tmp_lt_search_path_spec= lt_multi_os_dir=/`$CC $CPPFLAGS $CFLAGS $LDFLAGS -print-multi-os-directory 2>/dev/null` # ...but if some path component already ends with the multilib dir we assume # that all is fine and trust -print-search-dirs as is (GCC 4.2? or newer). case "$lt_multi_os_dir; $lt_search_path_spec " in "/; "* | "/.; "* | "/./; "* | *"$lt_multi_os_dir "* | *"$lt_multi_os_dir/ "*) lt_multi_os_dir= ;; esac for lt_sys_path in $lt_search_path_spec; do if test -d "$lt_sys_path$lt_multi_os_dir"; then lt_tmp_lt_search_path_spec="$lt_tmp_lt_search_path_spec $lt_sys_path$lt_multi_os_dir" elif test -n "$lt_multi_os_dir"; then test -d "$lt_sys_path" && \ lt_tmp_lt_search_path_spec="$lt_tmp_lt_search_path_spec $lt_sys_path" fi done lt_search_path_spec=`$ECHO "$lt_tmp_lt_search_path_spec" | awk ' BEGIN {RS = " "; FS = "/|\n";} { lt_foo = ""; lt_count = 0; for (lt_i = NF; lt_i > 0; lt_i--) { if ($lt_i != "" && $lt_i != ".") { if ($lt_i == "..") { lt_count++; } else { if (lt_count == 0) { lt_foo = "/" $lt_i lt_foo; } else { lt_count--; } } } } if (lt_foo != "") { lt_freq[lt_foo]++; } if (lt_freq[lt_foo] == 1) { print lt_foo; } }'` # AWK program above erroneously prepends '/' to C:/dos/paths # for these hosts. case $host_os in mingw* | cegcc*) lt_search_path_spec=`$ECHO "$lt_search_path_spec" |\ $SED 's|/\([A-Za-z]:\)|\1|g'` ;; esac sys_lib_search_path_spec=`$ECHO "$lt_search_path_spec" | $lt_NL2SP` else sys_lib_search_path_spec="/lib /usr/lib /usr/local/lib" fi library_names_spec= libname_spec='lib$name' soname_spec= shrext_cmds=.so postinstall_cmds= postuninstall_cmds= finish_cmds= finish_eval= shlibpath_var= shlibpath_overrides_runpath=unknown version_type=none dynamic_linker="$host_os ld.so" sys_lib_dlsearch_path_spec="/lib /usr/lib" need_lib_prefix=unknown hardcode_into_libs=no # when you set need_version to no, make sure it does not cause -set_version # flags to be left without arguments need_version=unknown case $host_os in aix3*) version_type=linux # correct to gnu/linux during the next big refactor library_names_spec='$libname$release$shared_ext$versuffix $libname.a' shlibpath_var=LIBPATH # AIX 3 has no versioning support, so we append a major version to the name. soname_spec='$libname$release$shared_ext$major' ;; aix[4-9]*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no hardcode_into_libs=yes if test ia64 = "$host_cpu"; then # AIX 5 supports IA64 library_names_spec='$libname$release$shared_ext$major $libname$release$shared_ext$versuffix $libname$shared_ext' shlibpath_var=LD_LIBRARY_PATH else # With GCC up to 2.95.x, collect2 would create an import file # for dependence libraries. The import file would start with # the line '#! .'. This would cause the generated library to # depend on '.', always an invalid library. This was fixed in # development snapshots of GCC prior to 3.0. case $host_os in aix4 | aix4.[01] | aix4.[01].*) if { echo '#if __GNUC__ > 2 || (__GNUC__ == 2 && __GNUC_MINOR__ >= 97)' echo ' yes ' echo '#endif'; } | $CC -E - | $GREP yes > /dev/null; then : else can_build_shared=no fi ;; esac # Using Import Files as archive members, it is possible to support # filename-based versioning of shared library archives on AIX. While # this would work for both with and without runtime linking, it will # prevent static linking of such archives. So we do filename-based # shared library versioning with .so extension only, which is used # when both runtime linking and shared linking is enabled. # Unfortunately, runtime linking may impact performance, so we do # not want this to be the default eventually. Also, we use the # versioned .so libs for executables only if there is the -brtl # linker flag in LDFLAGS as well, or --with-aix-soname=svr4 only. # To allow for filename-based versioning support, we need to create # libNAME.so.V as an archive file, containing: # *) an Import File, referring to the versioned filename of the # archive as well as the shared archive member, telling the # bitwidth (32 or 64) of that shared object, and providing the # list of exported symbols of that shared object, eventually # decorated with the 'weak' keyword # *) the shared object with the F_LOADONLY flag set, to really avoid # it being seen by the linker. # At run time we better use the real file rather than another symlink, # but for link time we create the symlink libNAME.so -> libNAME.so.V case $with_aix_soname,$aix_use_runtimelinking in # AIX (on Power*) has no versioning support, so currently we cannot hardcode correct # soname into executable. Probably we can add versioning support to # collect2, so additional links can be useful in future. aix,yes) # traditional libtool dynamic_linker='AIX unversionable lib.so' # If using run time linking (on AIX 4.2 or later) use lib.so # instead of lib.a to let people know that these are not # typical AIX shared libraries. library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' ;; aix,no) # traditional AIX only dynamic_linker='AIX lib.a(lib.so.V)' # We preserve .a as extension for shared libraries through AIX4.2 # and later when we are not doing run time linking. library_names_spec='$libname$release.a $libname.a' soname_spec='$libname$release$shared_ext$major' ;; svr4,*) # full svr4 only dynamic_linker="AIX lib.so.V($shared_archive_member_spec.o)" library_names_spec='$libname$release$shared_ext$major $libname$shared_ext' # We do not specify a path in Import Files, so LIBPATH fires. shlibpath_overrides_runpath=yes ;; *,yes) # both, prefer svr4 dynamic_linker="AIX lib.so.V($shared_archive_member_spec.o), lib.a(lib.so.V)" library_names_spec='$libname$release$shared_ext$major $libname$shared_ext' # unpreferred sharedlib libNAME.a needs extra handling postinstall_cmds='test -n "$linkname" || linkname="$realname"~func_stripname "" ".so" "$linkname"~$install_shared_prog "$dir/$func_stripname_result.$libext" "$destdir/$func_stripname_result.$libext"~test -z "$tstripme" || test -z "$striplib" || $striplib "$destdir/$func_stripname_result.$libext"' postuninstall_cmds='for n in $library_names $old_library; do :; done~func_stripname "" ".so" "$n"~test "$func_stripname_result" = "$n" || func_append rmfiles " $odir/$func_stripname_result.$libext"' # We do not specify a path in Import Files, so LIBPATH fires. shlibpath_overrides_runpath=yes ;; *,no) # both, prefer aix dynamic_linker="AIX lib.a(lib.so.V), lib.so.V($shared_archive_member_spec.o)" library_names_spec='$libname$release.a $libname.a' soname_spec='$libname$release$shared_ext$major' # unpreferred sharedlib libNAME.so.V and symlink libNAME.so need extra handling postinstall_cmds='test -z "$dlname" || $install_shared_prog $dir/$dlname $destdir/$dlname~test -z "$tstripme" || test -z "$striplib" || $striplib $destdir/$dlname~test -n "$linkname" || linkname=$realname~func_stripname "" ".a" "$linkname"~(cd "$destdir" && $LN_S -f $dlname $func_stripname_result.so)' postuninstall_cmds='test -z "$dlname" || func_append rmfiles " $odir/$dlname"~for n in $old_library $library_names; do :; done~func_stripname "" ".a" "$n"~func_append rmfiles " $odir/$func_stripname_result.so"' ;; esac shlibpath_var=LIBPATH fi ;; amigaos*) case $host_cpu in powerpc) # Since July 2007 AmigaOS4 officially supports .so libraries. # When compiling the executable, add -use-dynld -Lsobjs: to the compileline. library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' ;; m68k) library_names_spec='$libname.ixlibrary $libname.a' # Create ${libname}_ixlibrary.a entries in /sys/libs. finish_eval='for lib in `ls $libdir/*.ixlibrary 2>/dev/null`; do libname=`func_echo_all "$lib" | $SED '\''s%^.*/\([^/]*\)\.ixlibrary$%\1%'\''`; $RM /sys/libs/${libname}_ixlibrary.a; $show "cd /sys/libs && $LN_S $lib ${libname}_ixlibrary.a"; cd /sys/libs && $LN_S $lib ${libname}_ixlibrary.a || exit 1; done' ;; esac ;; beos*) library_names_spec='$libname$shared_ext' dynamic_linker="$host_os ld.so" shlibpath_var=LIBRARY_PATH ;; bsdi[45]*) version_type=linux # correct to gnu/linux during the next big refactor need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' finish_cmds='PATH="\$PATH:/sbin" ldconfig $libdir' shlibpath_var=LD_LIBRARY_PATH sys_lib_search_path_spec="/shlib /usr/lib /usr/X11/lib /usr/contrib/lib /lib /usr/local/lib" sys_lib_dlsearch_path_spec="/shlib /usr/lib /usr/local/lib" # the default ld.so.conf also contains /usr/contrib/lib and # /usr/X11R6/lib (/usr/X11 is a link to /usr/X11R6), but let us allow # libtool to hard-code these into programs ;; cygwin* | mingw* | pw32* | cegcc*) version_type=windows shrext_cmds=.dll need_version=no need_lib_prefix=no case $GCC,$cc_basename in yes,*) # gcc library_names_spec='$libname.dll.a' # DLL is installed to $(libdir)/../bin by postinstall_cmds postinstall_cmds='base_file=`basename \$file`~ dlpath=`$SHELL 2>&1 -c '\''. $dir/'\''\$base_file'\''i; echo \$dlname'\''`~ dldir=$destdir/`dirname \$dlpath`~ test -d \$dldir || mkdir -p \$dldir~ $install_prog $dir/$dlname \$dldir/$dlname~ chmod a+x \$dldir/$dlname~ if test -n '\''$stripme'\'' && test -n '\''$striplib'\''; then eval '\''$striplib \$dldir/$dlname'\'' || exit \$?; fi' postuninstall_cmds='dldll=`$SHELL 2>&1 -c '\''. $file; echo \$dlname'\''`~ dlpath=$dir/\$dldll~ $RM \$dlpath' shlibpath_overrides_runpath=yes case $host_os in cygwin*) # Cygwin DLLs use 'cyg' prefix rather than 'lib' soname_spec='`echo $libname | sed -e 's/^lib/cyg/'``echo $release | $SED -e 's/[.]/-/g'`$versuffix$shared_ext' sys_lib_search_path_spec="$sys_lib_search_path_spec /usr/lib/w32api" ;; mingw* | cegcc*) # MinGW DLLs use traditional 'lib' prefix soname_spec='$libname`echo $release | $SED -e 's/[.]/-/g'`$versuffix$shared_ext' ;; pw32*) # pw32 DLLs use 'pw' prefix rather than 'lib' library_names_spec='`echo $libname | sed -e 's/^lib/pw/'``echo $release | $SED -e 's/[.]/-/g'`$versuffix$shared_ext' ;; esac dynamic_linker='Win32 ld.exe' ;; *,cl*) # Native MSVC libname_spec='$name' soname_spec='$libname`echo $release | $SED -e 's/[.]/-/g'`$versuffix$shared_ext' library_names_spec='$libname.dll.lib' case $build_os in mingw*) sys_lib_search_path_spec= lt_save_ifs=$IFS IFS=';' for lt_path in $LIB do IFS=$lt_save_ifs # Let DOS variable expansion print the short 8.3 style file name. lt_path=`cd "$lt_path" 2>/dev/null && cmd //C "for %i in (".") do @echo %~si"` sys_lib_search_path_spec="$sys_lib_search_path_spec $lt_path" done IFS=$lt_save_ifs # Convert to MSYS style. sys_lib_search_path_spec=`$ECHO "$sys_lib_search_path_spec" | sed -e 's|\\\\|/|g' -e 's| \\([a-zA-Z]\\):| /\\1|g' -e 's|^ ||'` ;; cygwin*) # Convert to unix form, then to dos form, then back to unix form # but this time dos style (no spaces!) so that the unix form looks # like /cygdrive/c/PROGRA~1:/cygdr... sys_lib_search_path_spec=`cygpath --path --unix "$LIB"` sys_lib_search_path_spec=`cygpath --path --dos "$sys_lib_search_path_spec" 2>/dev/null` sys_lib_search_path_spec=`cygpath --path --unix "$sys_lib_search_path_spec" | $SED -e "s/$PATH_SEPARATOR/ /g"` ;; *) sys_lib_search_path_spec=$LIB if $ECHO "$sys_lib_search_path_spec" | $GREP ';[c-zC-Z]:/' >/dev/null; then # It is most probably a Windows format PATH. sys_lib_search_path_spec=`$ECHO "$sys_lib_search_path_spec" | $SED -e 's/;/ /g'` else sys_lib_search_path_spec=`$ECHO "$sys_lib_search_path_spec" | $SED -e "s/$PATH_SEPARATOR/ /g"` fi # FIXME: find the short name or the path components, as spaces are # common. (e.g. "Program Files" -> "PROGRA~1") ;; esac # DLL is installed to $(libdir)/../bin by postinstall_cmds postinstall_cmds='base_file=`basename \$file`~ dlpath=`$SHELL 2>&1 -c '\''. $dir/'\''\$base_file'\''i; echo \$dlname'\''`~ dldir=$destdir/`dirname \$dlpath`~ test -d \$dldir || mkdir -p \$dldir~ $install_prog $dir/$dlname \$dldir/$dlname' postuninstall_cmds='dldll=`$SHELL 2>&1 -c '\''. $file; echo \$dlname'\''`~ dlpath=$dir/\$dldll~ $RM \$dlpath' shlibpath_overrides_runpath=yes dynamic_linker='Win32 link.exe' ;; *) # Assume MSVC wrapper library_names_spec='$libname`echo $release | $SED -e 's/[.]/-/g'`$versuffix$shared_ext $libname.lib' dynamic_linker='Win32 ld.exe' ;; esac # FIXME: first we should search . and the directory the executable is in shlibpath_var=PATH ;; darwin* | rhapsody*) dynamic_linker="$host_os dyld" version_type=darwin need_lib_prefix=no need_version=no library_names_spec='$libname$release$major$shared_ext $libname$shared_ext' soname_spec='$libname$release$major$shared_ext' shlibpath_overrides_runpath=yes shlibpath_var=DYLD_LIBRARY_PATH shrext_cmds='`test .$module = .yes && echo .so || echo .dylib`' sys_lib_search_path_spec="$sys_lib_search_path_spec /usr/local/lib" sys_lib_dlsearch_path_spec='/usr/local/lib /lib /usr/lib' ;; dgux*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' shlibpath_var=LD_LIBRARY_PATH ;; freebsd* | dragonfly*) # DragonFly does not have aout. When/if they implement a new # versioning mechanism, adjust this. if test -x /usr/bin/objformat; then objformat=`/usr/bin/objformat` else case $host_os in freebsd[23].*) objformat=aout ;; *) objformat=elf ;; esac fi version_type=freebsd-$objformat case $version_type in freebsd-elf*) library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' need_version=no need_lib_prefix=no ;; freebsd-*) library_names_spec='$libname$release$shared_ext$versuffix $libname$shared_ext$versuffix' need_version=yes ;; esac shlibpath_var=LD_LIBRARY_PATH case $host_os in freebsd2.*) shlibpath_overrides_runpath=yes ;; freebsd3.[01]* | freebsdelf3.[01]*) shlibpath_overrides_runpath=yes hardcode_into_libs=yes ;; freebsd3.[2-9]* | freebsdelf3.[2-9]* | \ freebsd4.[0-5] | freebsdelf4.[0-5] | freebsd4.1.1 | freebsdelf4.1.1) shlibpath_overrides_runpath=no hardcode_into_libs=yes ;; *) # from 4.6 on, and DragonFly shlibpath_overrides_runpath=yes hardcode_into_libs=yes ;; esac ;; haiku*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no dynamic_linker="$host_os runtime_loader" library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' shlibpath_var=LIBRARY_PATH shlibpath_overrides_runpath=no sys_lib_dlsearch_path_spec='/boot/home/config/lib /boot/common/lib /boot/system/lib' hardcode_into_libs=yes ;; hpux9* | hpux10* | hpux11*) # Give a soname corresponding to the major version so that dld.sl refuses to # link against other versions. version_type=sunos need_lib_prefix=no need_version=no case $host_cpu in ia64*) shrext_cmds='.so' hardcode_into_libs=yes dynamic_linker="$host_os dld.so" shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes # Unless +noenvvar is specified. library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' if test 32 = "$HPUX_IA64_MODE"; then sys_lib_search_path_spec="/usr/lib/hpux32 /usr/local/lib/hpux32 /usr/local/lib" sys_lib_dlsearch_path_spec=/usr/lib/hpux32 else sys_lib_search_path_spec="/usr/lib/hpux64 /usr/local/lib/hpux64" sys_lib_dlsearch_path_spec=/usr/lib/hpux64 fi ;; hppa*64*) shrext_cmds='.sl' hardcode_into_libs=yes dynamic_linker="$host_os dld.sl" shlibpath_var=LD_LIBRARY_PATH # How should we handle SHLIB_PATH shlibpath_overrides_runpath=yes # Unless +noenvvar is specified. library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' sys_lib_search_path_spec="/usr/lib/pa20_64 /usr/ccs/lib/pa20_64" sys_lib_dlsearch_path_spec=$sys_lib_search_path_spec ;; *) shrext_cmds='.sl' dynamic_linker="$host_os dld.sl" shlibpath_var=SHLIB_PATH shlibpath_overrides_runpath=no # +s is required to enable SHLIB_PATH library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' ;; esac # HP-UX runs *really* slowly unless shared libraries are mode 555, ... postinstall_cmds='chmod 555 $lib' # or fails outright, so override atomically: install_override_mode=555 ;; interix[3-9]*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' dynamic_linker='Interix 3.x ld.so.1 (PE, like ELF)' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=no hardcode_into_libs=yes ;; irix5* | irix6* | nonstopux*) case $host_os in nonstopux*) version_type=nonstopux ;; *) if test yes = "$lt_cv_prog_gnu_ld"; then version_type=linux # correct to gnu/linux during the next big refactor else version_type=irix fi ;; esac need_lib_prefix=no need_version=no soname_spec='$libname$release$shared_ext$major' library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$release$shared_ext $libname$shared_ext' case $host_os in irix5* | nonstopux*) libsuff= shlibsuff= ;; *) case $LD in # libtool.m4 will add one of these switches to LD *-32|*"-32 "|*-melf32bsmip|*"-melf32bsmip ") libsuff= shlibsuff= libmagic=32-bit;; *-n32|*"-n32 "|*-melf32bmipn32|*"-melf32bmipn32 ") libsuff=32 shlibsuff=N32 libmagic=N32;; *-64|*"-64 "|*-melf64bmip|*"-melf64bmip ") libsuff=64 shlibsuff=64 libmagic=64-bit;; *) libsuff= shlibsuff= libmagic=never-match;; esac ;; esac shlibpath_var=LD_LIBRARY${shlibsuff}_PATH shlibpath_overrides_runpath=no sys_lib_search_path_spec="/usr/lib$libsuff /lib$libsuff /usr/local/lib$libsuff" sys_lib_dlsearch_path_spec="/usr/lib$libsuff /lib$libsuff" hardcode_into_libs=yes ;; # No shared lib support for Linux oldld, aout, or coff. linux*oldld* | linux*aout* | linux*coff*) dynamic_linker=no ;; linux*android*) version_type=none # Android doesn't support versioned libraries. need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext' soname_spec='$libname$release$shared_ext' finish_cmds= shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes # This implies no fast_install, which is unacceptable. # Some rework will be needed to allow for fast_install # before this can be enabled. hardcode_into_libs=yes dynamic_linker='Android linker' # Don't embed -rpath directories since the linker doesn't support them. hardcode_libdir_flag_spec='-L$libdir' ;; # This must be glibc/ELF. linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' finish_cmds='PATH="\$PATH:/sbin" ldconfig -n $libdir' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=no # Some binutils ld are patched to set DT_RUNPATH if ${lt_cv_shlibpath_overrides_runpath+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_shlibpath_overrides_runpath=no save_LDFLAGS=$LDFLAGS save_libdir=$libdir eval "libdir=/foo; wl=\"$lt_prog_compiler_wl\"; \ LDFLAGS=\"\$LDFLAGS $hardcode_libdir_flag_spec\"" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : if ($OBJDUMP -p conftest$ac_exeext) 2>/dev/null | grep "RUNPATH.*$libdir" >/dev/null; then : lt_cv_shlibpath_overrides_runpath=yes fi fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LDFLAGS=$save_LDFLAGS libdir=$save_libdir fi shlibpath_overrides_runpath=$lt_cv_shlibpath_overrides_runpath # This implies no fast_install, which is unacceptable. # Some rework will be needed to allow for fast_install # before this can be enabled. hardcode_into_libs=yes # Ideally, we could use ldconfig to report *all* directores which are # searched for libraries, however this is still not possible. Aside from not # being certain /sbin/ldconfig is available, command # 'ldconfig -N -X -v | grep ^/' on 64bit Fedora does not report /usr/lib64, # even though it is searched at run-time. Try to do the best guess by # appending ld.so.conf contents (and includes) to the search path. if test -f /etc/ld.so.conf; then lt_ld_extra=`awk '/^include / { system(sprintf("cd /etc; cat %s 2>/dev/null", \$2)); skip = 1; } { if (!skip) print \$0; skip = 0; }' < /etc/ld.so.conf | $SED -e 's/#.*//;/^[ ]*hwcap[ ]/d;s/[:, ]/ /g;s/=[^=]*$//;s/=[^= ]* / /g;s/"//g;/^$/d' | tr '\n' ' '` sys_lib_dlsearch_path_spec="/lib /usr/lib $lt_ld_extra" fi # We used to test for /lib/ld.so.1 and disable shared libraries on # powerpc, because MkLinux only supported shared libraries with the # GNU dynamic linker. Since this was broken with cross compilers, # most powerpc-linux boxes support dynamic linking these days and # people can always --disable-shared, the test was removed, and we # assume the GNU/Linux dynamic linker is in use. dynamic_linker='GNU/Linux ld.so' ;; netbsdelf*-gnu) version_type=linux need_lib_prefix=no need_version=no library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major ${libname}${shared_ext}' soname_spec='${libname}${release}${shared_ext}$major' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=no hardcode_into_libs=yes dynamic_linker='NetBSD ld.elf_so' ;; netbsd*) version_type=sunos need_lib_prefix=no need_version=no if echo __ELF__ | $CC -E - | $GREP __ELF__ >/dev/null; then library_names_spec='$libname$release$shared_ext$versuffix $libname$shared_ext$versuffix' finish_cmds='PATH="\$PATH:/sbin" ldconfig -m $libdir' dynamic_linker='NetBSD (a.out) ld.so' else library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' dynamic_linker='NetBSD ld.elf_so' fi shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes hardcode_into_libs=yes ;; newsos6) version_type=linux # correct to gnu/linux during the next big refactor library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes ;; *nto* | *qnx*) version_type=qnx need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=no hardcode_into_libs=yes dynamic_linker='ldqnx.so' ;; openbsd* | bitrig*) version_type=sunos sys_lib_dlsearch_path_spec=/usr/lib need_lib_prefix=no if test -z "`echo __ELF__ | $CC -E - | $GREP __ELF__`"; then need_version=no else need_version=yes fi library_names_spec='$libname$release$shared_ext$versuffix $libname$shared_ext$versuffix' finish_cmds='PATH="\$PATH:/sbin" ldconfig -m $libdir' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes ;; os2*) libname_spec='$name' version_type=windows shrext_cmds=.dll need_version=no need_lib_prefix=no # OS/2 can only load a DLL with a base name of 8 characters or less. soname_spec='`test -n "$os2dllname" && libname="$os2dllname"; v=$($ECHO $release$versuffix | tr -d .-); n=$($ECHO $libname | cut -b -$((8 - ${#v})) | tr . _); $ECHO $n$v`$shared_ext' library_names_spec='${libname}_dll.$libext' dynamic_linker='OS/2 ld.exe' shlibpath_var=BEGINLIBPATH sys_lib_search_path_spec="/lib /usr/lib /usr/local/lib" sys_lib_dlsearch_path_spec=$sys_lib_search_path_spec postinstall_cmds='base_file=`basename \$file`~ dlpath=`$SHELL 2>&1 -c '\''. $dir/'\''\$base_file'\''i; $ECHO \$dlname'\''`~ dldir=$destdir/`dirname \$dlpath`~ test -d \$dldir || mkdir -p \$dldir~ $install_prog $dir/$dlname \$dldir/$dlname~ chmod a+x \$dldir/$dlname~ if test -n '\''$stripme'\'' && test -n '\''$striplib'\''; then eval '\''$striplib \$dldir/$dlname'\'' || exit \$?; fi' postuninstall_cmds='dldll=`$SHELL 2>&1 -c '\''. $file; $ECHO \$dlname'\''`~ dlpath=$dir/\$dldll~ $RM \$dlpath' ;; osf3* | osf4* | osf5*) version_type=osf need_lib_prefix=no need_version=no soname_spec='$libname$release$shared_ext$major' library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' shlibpath_var=LD_LIBRARY_PATH sys_lib_search_path_spec="/usr/shlib /usr/ccs/lib /usr/lib/cmplrs/cc /usr/lib /usr/local/lib /var/shlib" sys_lib_dlsearch_path_spec=$sys_lib_search_path_spec ;; rdos*) dynamic_linker=no ;; solaris*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes hardcode_into_libs=yes # ldd complains unless libraries are executable postinstall_cmds='chmod +x $lib' ;; sunos4*) version_type=sunos library_names_spec='$libname$release$shared_ext$versuffix $libname$shared_ext$versuffix' finish_cmds='PATH="\$PATH:/usr/etc" ldconfig $libdir' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes if test yes = "$with_gnu_ld"; then need_lib_prefix=no fi need_version=yes ;; sysv4 | sysv4.3*) version_type=linux # correct to gnu/linux during the next big refactor library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' shlibpath_var=LD_LIBRARY_PATH case $host_vendor in sni) shlibpath_overrides_runpath=no need_lib_prefix=no runpath_var=LD_RUN_PATH ;; siemens) need_lib_prefix=no ;; motorola) need_lib_prefix=no need_version=no shlibpath_overrides_runpath=no sys_lib_search_path_spec='/lib /usr/lib /usr/ccs/lib' ;; esac ;; sysv4*MP*) if test -d /usr/nec; then version_type=linux # correct to gnu/linux during the next big refactor library_names_spec='$libname$shared_ext.$versuffix $libname$shared_ext.$major $libname$shared_ext' soname_spec='$libname$shared_ext.$major' shlibpath_var=LD_LIBRARY_PATH fi ;; sysv5* | sco3.2v5* | sco5v6* | unixware* | OpenUNIX* | sysv4*uw2*) version_type=sco need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes hardcode_into_libs=yes if test yes = "$with_gnu_ld"; then sys_lib_search_path_spec='/usr/local/lib /usr/gnu/lib /usr/ccs/lib /usr/lib /lib' else sys_lib_search_path_spec='/usr/ccs/lib /usr/lib' case $host_os in sco3.2v5*) sys_lib_search_path_spec="$sys_lib_search_path_spec /lib" ;; esac fi sys_lib_dlsearch_path_spec='/usr/lib' ;; tpf*) # TPF is a cross-target only. Preferred cross-host = GNU/Linux. version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=no hardcode_into_libs=yes ;; uts4*) version_type=linux # correct to gnu/linux during the next big refactor library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' shlibpath_var=LD_LIBRARY_PATH ;; *) dynamic_linker=no ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: result: $dynamic_linker" >&5 $as_echo "$dynamic_linker" >&6; } test no = "$dynamic_linker" && can_build_shared=no variables_saved_for_relink="PATH $shlibpath_var $runpath_var" if test yes = "$GCC"; then variables_saved_for_relink="$variables_saved_for_relink GCC_EXEC_PREFIX COMPILER_PATH LIBRARY_PATH" fi if test set = "${lt_cv_sys_lib_search_path_spec+set}"; then sys_lib_search_path_spec=$lt_cv_sys_lib_search_path_spec fi if test set = "${lt_cv_sys_lib_dlsearch_path_spec+set}"; then sys_lib_dlsearch_path_spec=$lt_cv_sys_lib_dlsearch_path_spec fi # remember unaugmented sys_lib_dlsearch_path content for libtool script decls... configure_time_dlsearch_path=$sys_lib_dlsearch_path_spec # ... but it needs LT_SYS_LIBRARY_PATH munging for other configure-time code func_munge_path_list sys_lib_dlsearch_path_spec "$LT_SYS_LIBRARY_PATH" # to be used as default LT_SYS_LIBRARY_PATH value in generated libtool configure_time_lt_sys_library_path=$LT_SYS_LIBRARY_PATH { $as_echo "$as_me:${as_lineno-$LINENO}: checking how to hardcode library paths into programs" >&5 $as_echo_n "checking how to hardcode library paths into programs... " >&6; } hardcode_action= if test -n "$hardcode_libdir_flag_spec" || test -n "$runpath_var" || test yes = "$hardcode_automatic"; then # We can hardcode non-existent directories. if test no != "$hardcode_direct" && # If the only mechanism to avoid hardcoding is shlibpath_var, we # have to relink, otherwise we might link with an installed library # when we should be linking with a yet-to-be-installed one ## test no != "$_LT_TAGVAR(hardcode_shlibpath_var, )" && test no != "$hardcode_minus_L"; then # Linking always hardcodes the temporary library directory. hardcode_action=relink else # We can link without hardcoding, and we can hardcode nonexisting dirs. hardcode_action=immediate fi else # We cannot hardcode anything, or else we can only hardcode existing # directories. hardcode_action=unsupported fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $hardcode_action" >&5 $as_echo "$hardcode_action" >&6; } if test relink = "$hardcode_action" || test yes = "$inherit_rpath"; then # Fast installation is not supported enable_fast_install=no elif test yes = "$shlibpath_overrides_runpath" || test no = "$enable_shared"; then # Fast installation is not necessary enable_fast_install=needless fi if test yes != "$enable_dlopen"; then enable_dlopen=unknown enable_dlopen_self=unknown enable_dlopen_self_static=unknown else lt_cv_dlopen=no lt_cv_dlopen_libs= case $host_os in beos*) lt_cv_dlopen=load_add_on lt_cv_dlopen_libs= lt_cv_dlopen_self=yes ;; mingw* | pw32* | cegcc*) lt_cv_dlopen=LoadLibrary lt_cv_dlopen_libs= ;; cygwin*) lt_cv_dlopen=dlopen lt_cv_dlopen_libs= ;; darwin*) # if libdl is installed we need to link against it { $as_echo "$as_me:${as_lineno-$LINENO}: checking for dlopen in -ldl" >&5 $as_echo_n "checking for dlopen in -ldl... " >&6; } if ${ac_cv_lib_dl_dlopen+:} false; then : $as_echo_n "(cached) " >&6 else ac_check_lib_save_LIBS=$LIBS LIBS="-ldl $LIBS" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ /* Override any GCC internal prototype to avoid an error. Use char because int might match the return type of a GCC builtin and then its argument prototype would still apply. */ #ifdef __cplusplus extern "C" #endif char dlopen (); int main () { return dlopen (); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ac_cv_lib_dl_dlopen=yes else ac_cv_lib_dl_dlopen=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_dl_dlopen" >&5 $as_echo "$ac_cv_lib_dl_dlopen" >&6; } if test "x$ac_cv_lib_dl_dlopen" = xyes; then : lt_cv_dlopen=dlopen lt_cv_dlopen_libs=-ldl else lt_cv_dlopen=dyld lt_cv_dlopen_libs= lt_cv_dlopen_self=yes fi ;; tpf*) # Don't try to run any link tests for TPF. We know it's impossible # because TPF is a cross-compiler, and we know how we open DSOs. lt_cv_dlopen=dlopen lt_cv_dlopen_libs= lt_cv_dlopen_self=no ;; *) ac_fn_c_check_func "$LINENO" "shl_load" "ac_cv_func_shl_load" if test "x$ac_cv_func_shl_load" = xyes; then : lt_cv_dlopen=shl_load else { $as_echo "$as_me:${as_lineno-$LINENO}: checking for shl_load in -ldld" >&5 $as_echo_n "checking for shl_load in -ldld... " >&6; } if ${ac_cv_lib_dld_shl_load+:} false; then : $as_echo_n "(cached) " >&6 else ac_check_lib_save_LIBS=$LIBS LIBS="-ldld $LIBS" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ /* Override any GCC internal prototype to avoid an error. Use char because int might match the return type of a GCC builtin and then its argument prototype would still apply. */ #ifdef __cplusplus extern "C" #endif char shl_load (); int main () { return shl_load (); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ac_cv_lib_dld_shl_load=yes else ac_cv_lib_dld_shl_load=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_dld_shl_load" >&5 $as_echo "$ac_cv_lib_dld_shl_load" >&6; } if test "x$ac_cv_lib_dld_shl_load" = xyes; then : lt_cv_dlopen=shl_load lt_cv_dlopen_libs=-ldld else ac_fn_c_check_func "$LINENO" "dlopen" "ac_cv_func_dlopen" if test "x$ac_cv_func_dlopen" = xyes; then : lt_cv_dlopen=dlopen else { $as_echo "$as_me:${as_lineno-$LINENO}: checking for dlopen in -ldl" >&5 $as_echo_n "checking for dlopen in -ldl... " >&6; } if ${ac_cv_lib_dl_dlopen+:} false; then : $as_echo_n "(cached) " >&6 else ac_check_lib_save_LIBS=$LIBS LIBS="-ldl $LIBS" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ /* Override any GCC internal prototype to avoid an error. Use char because int might match the return type of a GCC builtin and then its argument prototype would still apply. */ #ifdef __cplusplus extern "C" #endif char dlopen (); int main () { return dlopen (); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ac_cv_lib_dl_dlopen=yes else ac_cv_lib_dl_dlopen=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_dl_dlopen" >&5 $as_echo "$ac_cv_lib_dl_dlopen" >&6; } if test "x$ac_cv_lib_dl_dlopen" = xyes; then : lt_cv_dlopen=dlopen lt_cv_dlopen_libs=-ldl else { $as_echo "$as_me:${as_lineno-$LINENO}: checking for dlopen in -lsvld" >&5 $as_echo_n "checking for dlopen in -lsvld... " >&6; } if ${ac_cv_lib_svld_dlopen+:} false; then : $as_echo_n "(cached) " >&6 else ac_check_lib_save_LIBS=$LIBS LIBS="-lsvld $LIBS" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ /* Override any GCC internal prototype to avoid an error. Use char because int might match the return type of a GCC builtin and then its argument prototype would still apply. */ #ifdef __cplusplus extern "C" #endif char dlopen (); int main () { return dlopen (); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ac_cv_lib_svld_dlopen=yes else ac_cv_lib_svld_dlopen=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_svld_dlopen" >&5 $as_echo "$ac_cv_lib_svld_dlopen" >&6; } if test "x$ac_cv_lib_svld_dlopen" = xyes; then : lt_cv_dlopen=dlopen lt_cv_dlopen_libs=-lsvld else { $as_echo "$as_me:${as_lineno-$LINENO}: checking for dld_link in -ldld" >&5 $as_echo_n "checking for dld_link in -ldld... " >&6; } if ${ac_cv_lib_dld_dld_link+:} false; then : $as_echo_n "(cached) " >&6 else ac_check_lib_save_LIBS=$LIBS LIBS="-ldld $LIBS" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ /* Override any GCC internal prototype to avoid an error. Use char because int might match the return type of a GCC builtin and then its argument prototype would still apply. */ #ifdef __cplusplus extern "C" #endif char dld_link (); int main () { return dld_link (); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ac_cv_lib_dld_dld_link=yes else ac_cv_lib_dld_dld_link=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_dld_dld_link" >&5 $as_echo "$ac_cv_lib_dld_dld_link" >&6; } if test "x$ac_cv_lib_dld_dld_link" = xyes; then : lt_cv_dlopen=dld_link lt_cv_dlopen_libs=-ldld fi fi fi fi fi fi ;; esac if test no = "$lt_cv_dlopen"; then enable_dlopen=no else enable_dlopen=yes fi case $lt_cv_dlopen in dlopen) save_CPPFLAGS=$CPPFLAGS test yes = "$ac_cv_header_dlfcn_h" && CPPFLAGS="$CPPFLAGS -DHAVE_DLFCN_H" save_LDFLAGS=$LDFLAGS wl=$lt_prog_compiler_wl eval LDFLAGS=\"\$LDFLAGS $export_dynamic_flag_spec\" save_LIBS=$LIBS LIBS="$lt_cv_dlopen_libs $LIBS" { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether a program can dlopen itself" >&5 $as_echo_n "checking whether a program can dlopen itself... " >&6; } if ${lt_cv_dlopen_self+:} false; then : $as_echo_n "(cached) " >&6 else if test yes = "$cross_compiling"; then : lt_cv_dlopen_self=cross else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF #line $LINENO "configure" #include "confdefs.h" #if HAVE_DLFCN_H #include #endif #include #ifdef RTLD_GLOBAL # define LT_DLGLOBAL RTLD_GLOBAL #else # ifdef DL_GLOBAL # define LT_DLGLOBAL DL_GLOBAL # else # define LT_DLGLOBAL 0 # endif #endif /* We may have to define LT_DLLAZY_OR_NOW in the command line if we find out it does not work in some platform. */ #ifndef LT_DLLAZY_OR_NOW # ifdef RTLD_LAZY # define LT_DLLAZY_OR_NOW RTLD_LAZY # else # ifdef DL_LAZY # define LT_DLLAZY_OR_NOW DL_LAZY # else # ifdef RTLD_NOW # define LT_DLLAZY_OR_NOW RTLD_NOW # else # ifdef DL_NOW # define LT_DLLAZY_OR_NOW DL_NOW # else # define LT_DLLAZY_OR_NOW 0 # endif # endif # endif # endif #endif /* When -fvisibility=hidden is used, assume the code has been annotated correspondingly for the symbols needed. */ #if defined __GNUC__ && (((__GNUC__ == 3) && (__GNUC_MINOR__ >= 3)) || (__GNUC__ > 3)) int fnord () __attribute__((visibility("default"))); #endif int fnord () { return 42; } int main () { void *self = dlopen (0, LT_DLGLOBAL|LT_DLLAZY_OR_NOW); int status = $lt_dlunknown; if (self) { if (dlsym (self,"fnord")) status = $lt_dlno_uscore; else { if (dlsym( self,"_fnord")) status = $lt_dlneed_uscore; else puts (dlerror ()); } /* dlclose (self); */ } else puts (dlerror ()); return status; } _LT_EOF if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_link\""; } >&5 (eval $ac_link) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } && test -s "conftest$ac_exeext" 2>/dev/null; then (./conftest; exit; ) >&5 2>/dev/null lt_status=$? case x$lt_status in x$lt_dlno_uscore) lt_cv_dlopen_self=yes ;; x$lt_dlneed_uscore) lt_cv_dlopen_self=yes ;; x$lt_dlunknown|x*) lt_cv_dlopen_self=no ;; esac else : # compilation failed lt_cv_dlopen_self=no fi fi rm -fr conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_dlopen_self" >&5 $as_echo "$lt_cv_dlopen_self" >&6; } if test yes = "$lt_cv_dlopen_self"; then wl=$lt_prog_compiler_wl eval LDFLAGS=\"\$LDFLAGS $lt_prog_compiler_static\" { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether a statically linked program can dlopen itself" >&5 $as_echo_n "checking whether a statically linked program can dlopen itself... " >&6; } if ${lt_cv_dlopen_self_static+:} false; then : $as_echo_n "(cached) " >&6 else if test yes = "$cross_compiling"; then : lt_cv_dlopen_self_static=cross else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF #line $LINENO "configure" #include "confdefs.h" #if HAVE_DLFCN_H #include #endif #include #ifdef RTLD_GLOBAL # define LT_DLGLOBAL RTLD_GLOBAL #else # ifdef DL_GLOBAL # define LT_DLGLOBAL DL_GLOBAL # else # define LT_DLGLOBAL 0 # endif #endif /* We may have to define LT_DLLAZY_OR_NOW in the command line if we find out it does not work in some platform. */ #ifndef LT_DLLAZY_OR_NOW # ifdef RTLD_LAZY # define LT_DLLAZY_OR_NOW RTLD_LAZY # else # ifdef DL_LAZY # define LT_DLLAZY_OR_NOW DL_LAZY # else # ifdef RTLD_NOW # define LT_DLLAZY_OR_NOW RTLD_NOW # else # ifdef DL_NOW # define LT_DLLAZY_OR_NOW DL_NOW # else # define LT_DLLAZY_OR_NOW 0 # endif # endif # endif # endif #endif /* When -fvisibility=hidden is used, assume the code has been annotated correspondingly for the symbols needed. */ #if defined __GNUC__ && (((__GNUC__ == 3) && (__GNUC_MINOR__ >= 3)) || (__GNUC__ > 3)) int fnord () __attribute__((visibility("default"))); #endif int fnord () { return 42; } int main () { void *self = dlopen (0, LT_DLGLOBAL|LT_DLLAZY_OR_NOW); int status = $lt_dlunknown; if (self) { if (dlsym (self,"fnord")) status = $lt_dlno_uscore; else { if (dlsym( self,"_fnord")) status = $lt_dlneed_uscore; else puts (dlerror ()); } /* dlclose (self); */ } else puts (dlerror ()); return status; } _LT_EOF if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_link\""; } >&5 (eval $ac_link) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } && test -s "conftest$ac_exeext" 2>/dev/null; then (./conftest; exit; ) >&5 2>/dev/null lt_status=$? case x$lt_status in x$lt_dlno_uscore) lt_cv_dlopen_self_static=yes ;; x$lt_dlneed_uscore) lt_cv_dlopen_self_static=yes ;; x$lt_dlunknown|x*) lt_cv_dlopen_self_static=no ;; esac else : # compilation failed lt_cv_dlopen_self_static=no fi fi rm -fr conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_dlopen_self_static" >&5 $as_echo "$lt_cv_dlopen_self_static" >&6; } fi CPPFLAGS=$save_CPPFLAGS LDFLAGS=$save_LDFLAGS LIBS=$save_LIBS ;; esac case $lt_cv_dlopen_self in yes|no) enable_dlopen_self=$lt_cv_dlopen_self ;; *) enable_dlopen_self=unknown ;; esac case $lt_cv_dlopen_self_static in yes|no) enable_dlopen_self_static=$lt_cv_dlopen_self_static ;; *) enable_dlopen_self_static=unknown ;; esac fi striplib= old_striplib= { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether stripping libraries is possible" >&5 $as_echo_n "checking whether stripping libraries is possible... " >&6; } if test -n "$STRIP" && $STRIP -V 2>&1 | $GREP "GNU strip" >/dev/null; then test -z "$old_striplib" && old_striplib="$STRIP --strip-debug" test -z "$striplib" && striplib="$STRIP --strip-unneeded" { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } else # FIXME - insert some real tests, host_os isn't really good enough case $host_os in darwin*) if test -n "$STRIP"; then striplib="$STRIP -x" old_striplib="$STRIP -S" { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi ;; *) { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } ;; esac fi # Report what library types will actually be built { $as_echo "$as_me:${as_lineno-$LINENO}: checking if libtool supports shared libraries" >&5 $as_echo_n "checking if libtool supports shared libraries... " >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: result: $can_build_shared" >&5 $as_echo "$can_build_shared" >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build shared libraries" >&5 $as_echo_n "checking whether to build shared libraries... " >&6; } test no = "$can_build_shared" && enable_shared=no # On AIX, shared libraries and static libraries use the same namespace, and # are all built from PIC. case $host_os in aix3*) test yes = "$enable_shared" && enable_static=no if test -n "$RANLIB"; then archive_cmds="$archive_cmds~\$RANLIB \$lib" postinstall_cmds='$RANLIB $lib' fi ;; aix[4-9]*) if test ia64 != "$host_cpu"; then case $enable_shared,$with_aix_soname,$aix_use_runtimelinking in yes,aix,yes) ;; # shared object as lib.so file only yes,svr4,*) ;; # shared object as lib.so archive member only yes,*) enable_static=no ;; # shared object in lib.a archive as well esac fi ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: result: $enable_shared" >&5 $as_echo "$enable_shared" >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build static libraries" >&5 $as_echo_n "checking whether to build static libraries... " >&6; } # Make sure either enable_shared or enable_static is yes. test yes = "$enable_shared" || enable_static=yes { $as_echo "$as_me:${as_lineno-$LINENO}: result: $enable_static" >&5 $as_echo "$enable_static" >&6; } fi ac_ext=cpp ac_cpp='$CXXCPP $CPPFLAGS' ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_cxx_compiler_gnu CC=$lt_save_CC if test -n "$CXX" && ( test no != "$CXX" && ( (test g++ = "$CXX" && `g++ -v >/dev/null 2>&1` ) || (test g++ != "$CXX"))); then ac_ext=cpp ac_cpp='$CXXCPP $CPPFLAGS' ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_cxx_compiler_gnu { $as_echo "$as_me:${as_lineno-$LINENO}: checking how to run the C++ preprocessor" >&5 $as_echo_n "checking how to run the C++ preprocessor... " >&6; } if test -z "$CXXCPP"; then if ${ac_cv_prog_CXXCPP+:} false; then : $as_echo_n "(cached) " >&6 else # Double quotes because CXXCPP needs to be expanded for CXXCPP in "$CXX -E" "/lib/cpp" do ac_preproc_ok=false for ac_cxx_preproc_warn_flag in '' yes do # Use a header file that comes with gcc, so configuring glibc # with a fresh cross-compiler works. # Prefer to if __STDC__ is defined, since # exists even on freestanding compilers. # On the NeXT, cc -E runs the code through the compiler's parser, # not just through cpp. "Syntax error" is here to catch this case. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #ifdef __STDC__ # include #else # include #endif Syntax error _ACEOF if ac_fn_cxx_try_cpp "$LINENO"; then : else # Broken: fails on valid input. continue fi rm -f conftest.err conftest.i conftest.$ac_ext # OK, works on sane cases. Now check whether nonexistent headers # can be detected and how. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include _ACEOF if ac_fn_cxx_try_cpp "$LINENO"; then : # Broken: success on invalid input. continue else # Passes both tests. ac_preproc_ok=: break fi rm -f conftest.err conftest.i conftest.$ac_ext done # Because of `break', _AC_PREPROC_IFELSE's cleaning code was skipped. rm -f conftest.i conftest.err conftest.$ac_ext if $ac_preproc_ok; then : break fi done ac_cv_prog_CXXCPP=$CXXCPP fi CXXCPP=$ac_cv_prog_CXXCPP else ac_cv_prog_CXXCPP=$CXXCPP fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $CXXCPP" >&5 $as_echo "$CXXCPP" >&6; } ac_preproc_ok=false for ac_cxx_preproc_warn_flag in '' yes do # Use a header file that comes with gcc, so configuring glibc # with a fresh cross-compiler works. # Prefer to if __STDC__ is defined, since # exists even on freestanding compilers. # On the NeXT, cc -E runs the code through the compiler's parser, # not just through cpp. "Syntax error" is here to catch this case. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #ifdef __STDC__ # include #else # include #endif Syntax error _ACEOF if ac_fn_cxx_try_cpp "$LINENO"; then : else # Broken: fails on valid input. continue fi rm -f conftest.err conftest.i conftest.$ac_ext # OK, works on sane cases. Now check whether nonexistent headers # can be detected and how. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include _ACEOF if ac_fn_cxx_try_cpp "$LINENO"; then : # Broken: success on invalid input. continue else # Passes both tests. ac_preproc_ok=: break fi rm -f conftest.err conftest.i conftest.$ac_ext done # Because of `break', _AC_PREPROC_IFELSE's cleaning code was skipped. rm -f conftest.i conftest.err conftest.$ac_ext if $ac_preproc_ok; then : else { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 $as_echo "$as_me: error: in \`$ac_pwd':" >&2;} as_fn_error $? "C++ preprocessor \"$CXXCPP\" fails sanity check See \`config.log' for more details" "$LINENO" 5; } fi ac_ext=cpp ac_cpp='$CXXCPP $CPPFLAGS' ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_cxx_compiler_gnu else _lt_caught_CXX_error=yes fi ac_ext=cpp ac_cpp='$CXXCPP $CPPFLAGS' ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_cxx_compiler_gnu archive_cmds_need_lc_CXX=no allow_undefined_flag_CXX= always_export_symbols_CXX=no archive_expsym_cmds_CXX= compiler_needs_object_CXX=no export_dynamic_flag_spec_CXX= hardcode_direct_CXX=no hardcode_direct_absolute_CXX=no hardcode_libdir_flag_spec_CXX= hardcode_libdir_separator_CXX= hardcode_minus_L_CXX=no hardcode_shlibpath_var_CXX=unsupported hardcode_automatic_CXX=no inherit_rpath_CXX=no module_cmds_CXX= module_expsym_cmds_CXX= link_all_deplibs_CXX=unknown old_archive_cmds_CXX=$old_archive_cmds reload_flag_CXX=$reload_flag reload_cmds_CXX=$reload_cmds no_undefined_flag_CXX= whole_archive_flag_spec_CXX= enable_shared_with_static_runtimes_CXX=no # Source file extension for C++ test sources. ac_ext=cpp # Object file extension for compiled C++ test sources. objext=o objext_CXX=$objext # No sense in running all these tests if we already determined that # the CXX compiler isn't working. Some variables (like enable_shared) # are currently assumed to apply to all compilers on this platform, # and will be corrupted by setting them based on a non-working compiler. if test yes != "$_lt_caught_CXX_error"; then # Code to be used in simple compile tests lt_simple_compile_test_code="int some_variable = 0;" # Code to be used in simple link tests lt_simple_link_test_code='int main(int, char *[]) { return(0); }' # ltmain only uses $CC for tagged configurations so make sure $CC is set. # If no C compiler was specified, use CC. LTCC=${LTCC-"$CC"} # If no C compiler flags were specified, use CFLAGS. LTCFLAGS=${LTCFLAGS-"$CFLAGS"} # Allow CC to be a program name with arguments. compiler=$CC # save warnings/boilerplate of simple test code ac_outfile=conftest.$ac_objext echo "$lt_simple_compile_test_code" >conftest.$ac_ext eval "$ac_compile" 2>&1 >/dev/null | $SED '/^$/d; /^ *+/d' >conftest.err _lt_compiler_boilerplate=`cat conftest.err` $RM conftest* ac_outfile=conftest.$ac_objext echo "$lt_simple_link_test_code" >conftest.$ac_ext eval "$ac_link" 2>&1 >/dev/null | $SED '/^$/d; /^ *+/d' >conftest.err _lt_linker_boilerplate=`cat conftest.err` $RM -r conftest* # Allow CC to be a program name with arguments. lt_save_CC=$CC lt_save_CFLAGS=$CFLAGS lt_save_LD=$LD lt_save_GCC=$GCC GCC=$GXX lt_save_with_gnu_ld=$with_gnu_ld lt_save_path_LD=$lt_cv_path_LD if test -n "${lt_cv_prog_gnu_ldcxx+set}"; then lt_cv_prog_gnu_ld=$lt_cv_prog_gnu_ldcxx else $as_unset lt_cv_prog_gnu_ld fi if test -n "${lt_cv_path_LDCXX+set}"; then lt_cv_path_LD=$lt_cv_path_LDCXX else $as_unset lt_cv_path_LD fi test -z "${LDCXX+set}" || LD=$LDCXX CC=${CXX-"c++"} CFLAGS=$CXXFLAGS compiler=$CC compiler_CXX=$CC func_cc_basename $compiler cc_basename=$func_cc_basename_result if test -n "$compiler"; then # We don't want -fno-exception when compiling C++ code, so set the # no_builtin_flag separately if test yes = "$GXX"; then lt_prog_compiler_no_builtin_flag_CXX=' -fno-builtin' else lt_prog_compiler_no_builtin_flag_CXX= fi if test yes = "$GXX"; then # Set up default GNU C++ configuration # Check whether --with-gnu-ld was given. if test "${with_gnu_ld+set}" = set; then : withval=$with_gnu_ld; test no = "$withval" || with_gnu_ld=yes else with_gnu_ld=no fi ac_prog=ld if test yes = "$GCC"; then # Check if gcc -print-prog-name=ld gives a path. { $as_echo "$as_me:${as_lineno-$LINENO}: checking for ld used by $CC" >&5 $as_echo_n "checking for ld used by $CC... " >&6; } case $host in *-*-mingw*) # gcc leaves a trailing carriage return, which upsets mingw ac_prog=`($CC -print-prog-name=ld) 2>&5 | tr -d '\015'` ;; *) ac_prog=`($CC -print-prog-name=ld) 2>&5` ;; esac case $ac_prog in # Accept absolute paths. [\\/]* | ?:[\\/]*) re_direlt='/[^/][^/]*/\.\./' # Canonicalize the pathname of ld ac_prog=`$ECHO "$ac_prog"| $SED 's%\\\\%/%g'` while $ECHO "$ac_prog" | $GREP "$re_direlt" > /dev/null 2>&1; do ac_prog=`$ECHO $ac_prog| $SED "s%$re_direlt%/%"` done test -z "$LD" && LD=$ac_prog ;; "") # If it fails, then pretend we aren't using GCC. ac_prog=ld ;; *) # If it is relative, then search for the first ld in PATH. with_gnu_ld=unknown ;; esac elif test yes = "$with_gnu_ld"; then { $as_echo "$as_me:${as_lineno-$LINENO}: checking for GNU ld" >&5 $as_echo_n "checking for GNU ld... " >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: checking for non-GNU ld" >&5 $as_echo_n "checking for non-GNU ld... " >&6; } fi if ${lt_cv_path_LD+:} false; then : $as_echo_n "(cached) " >&6 else if test -z "$LD"; then lt_save_ifs=$IFS; IFS=$PATH_SEPARATOR for ac_dir in $PATH; do IFS=$lt_save_ifs test -z "$ac_dir" && ac_dir=. if test -f "$ac_dir/$ac_prog" || test -f "$ac_dir/$ac_prog$ac_exeext"; then lt_cv_path_LD=$ac_dir/$ac_prog # Check to see if the program is GNU ld. I'd rather use --version, # but apparently some variants of GNU ld only accept -v. # Break only if it was the GNU/non-GNU ld that we prefer. case `"$lt_cv_path_LD" -v 2>&1 &5 $as_echo "$LD" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi test -z "$LD" && as_fn_error $? "no acceptable ld found in \$PATH" "$LINENO" 5 { $as_echo "$as_me:${as_lineno-$LINENO}: checking if the linker ($LD) is GNU ld" >&5 $as_echo_n "checking if the linker ($LD) is GNU ld... " >&6; } if ${lt_cv_prog_gnu_ld+:} false; then : $as_echo_n "(cached) " >&6 else # I'd rather use --version here, but apparently some GNU lds only accept -v. case `$LD -v 2>&1 &5 $as_echo "$lt_cv_prog_gnu_ld" >&6; } with_gnu_ld=$lt_cv_prog_gnu_ld # Check if GNU C++ uses GNU ld as the underlying linker, since the # archiving commands below assume that GNU ld is being used. if test yes = "$with_gnu_ld"; then archive_cmds_CXX='$CC $pic_flag -shared -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname -o $lib' archive_expsym_cmds_CXX='$CC $pic_flag -shared -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' hardcode_libdir_flag_spec_CXX='$wl-rpath $wl$libdir' export_dynamic_flag_spec_CXX='$wl--export-dynamic' # If archive_cmds runs LD, not CC, wlarc should be empty # XXX I think wlarc can be eliminated in ltcf-cxx, but I need to # investigate it a little bit more. (MM) wlarc='$wl' # ancient GNU ld didn't support --whole-archive et. al. if eval "`$CC -print-prog-name=ld` --help 2>&1" | $GREP 'no-whole-archive' > /dev/null; then whole_archive_flag_spec_CXX=$wlarc'--whole-archive$convenience '$wlarc'--no-whole-archive' else whole_archive_flag_spec_CXX= fi else with_gnu_ld=no wlarc= # A generic and very simple default shared library creation # command for GNU C++ for the case where it uses the native # linker, instead of GNU ld. If possible, this setting should # overridden to take advantage of the native linker features on # the platform it is being used on. archive_cmds_CXX='$CC -shared -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags -o $lib' fi # Commands to make compiler produce verbose output that lists # what "hidden" libraries, object files and flags are used when # linking a shared library. output_verbose_link_cmd='$CC -shared $CFLAGS -v conftest.$objext 2>&1 | $GREP -v "^Configured with:" | $GREP "\-L"' else GXX=no with_gnu_ld=no wlarc= fi # PORTME: fill in a description of your system's C++ link characteristics { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether the $compiler linker ($LD) supports shared libraries" >&5 $as_echo_n "checking whether the $compiler linker ($LD) supports shared libraries... " >&6; } ld_shlibs_CXX=yes case $host_os in aix3*) # FIXME: insert proper C++ library support ld_shlibs_CXX=no ;; aix[4-9]*) if test ia64 = "$host_cpu"; then # On IA64, the linker does run time linking by default, so we don't # have to do anything special. aix_use_runtimelinking=no exp_sym_flag='-Bexport' no_entry_flag= else aix_use_runtimelinking=no # Test if we are trying to use run time linking or normal # AIX style linking. If -brtl is somewhere in LDFLAGS, we # have runtime linking enabled, and use it for executables. # For shared libraries, we enable/disable runtime linking # depending on the kind of the shared library created - # when "with_aix_soname,aix_use_runtimelinking" is: # "aix,no" lib.a(lib.so.V) shared, rtl:no, for executables # "aix,yes" lib.so shared, rtl:yes, for executables # lib.a static archive # "both,no" lib.so.V(shr.o) shared, rtl:yes # lib.a(lib.so.V) shared, rtl:no, for executables # "both,yes" lib.so.V(shr.o) shared, rtl:yes, for executables # lib.a(lib.so.V) shared, rtl:no # "svr4,*" lib.so.V(shr.o) shared, rtl:yes, for executables # lib.a static archive case $host_os in aix4.[23]|aix4.[23].*|aix[5-9]*) for ld_flag in $LDFLAGS; do case $ld_flag in *-brtl*) aix_use_runtimelinking=yes break ;; esac done if test svr4,no = "$with_aix_soname,$aix_use_runtimelinking"; then # With aix-soname=svr4, we create the lib.so.V shared archives only, # so we don't have lib.a shared libs to link our executables. # We have to force runtime linking in this case. aix_use_runtimelinking=yes LDFLAGS="$LDFLAGS -Wl,-brtl" fi ;; esac exp_sym_flag='-bexport' no_entry_flag='-bnoentry' fi # When large executables or shared objects are built, AIX ld can # have problems creating the table of contents. If linking a library # or program results in "error TOC overflow" add -mminimal-toc to # CXXFLAGS/CFLAGS for g++/gcc. In the cases where that is not # enough to fix the problem, add -Wl,-bbigtoc to LDFLAGS. archive_cmds_CXX='' hardcode_direct_CXX=yes hardcode_direct_absolute_CXX=yes hardcode_libdir_separator_CXX=':' link_all_deplibs_CXX=yes file_list_spec_CXX='$wl-f,' case $with_aix_soname,$aix_use_runtimelinking in aix,*) ;; # no import file svr4,* | *,yes) # use import file # The Import File defines what to hardcode. hardcode_direct_CXX=no hardcode_direct_absolute_CXX=no ;; esac if test yes = "$GXX"; then case $host_os in aix4.[012]|aix4.[012].*) # We only want to do this on AIX 4.2 and lower, the check # below for broken collect2 doesn't work under 4.3+ collect2name=`$CC -print-prog-name=collect2` if test -f "$collect2name" && strings "$collect2name" | $GREP resolve_lib_name >/dev/null then # We have reworked collect2 : else # We have old collect2 hardcode_direct_CXX=unsupported # It fails to find uninstalled libraries when the uninstalled # path is not listed in the libpath. Setting hardcode_minus_L # to unsupported forces relinking hardcode_minus_L_CXX=yes hardcode_libdir_flag_spec_CXX='-L$libdir' hardcode_libdir_separator_CXX= fi esac shared_flag='-shared' if test yes = "$aix_use_runtimelinking"; then shared_flag=$shared_flag' $wl-G' fi # Need to ensure runtime linking is disabled for the traditional # shared library, or the linker may eventually find shared libraries # /with/ Import File - we do not want to mix them. shared_flag_aix='-shared' shared_flag_svr4='-shared $wl-G' else # not using gcc if test ia64 = "$host_cpu"; then # VisualAge C++, Version 5.5 for AIX 5L for IA-64, Beta 3 Release # chokes on -Wl,-G. The following line is correct: shared_flag='-G' else if test yes = "$aix_use_runtimelinking"; then shared_flag='$wl-G' else shared_flag='$wl-bM:SRE' fi shared_flag_aix='$wl-bM:SRE' shared_flag_svr4='$wl-G' fi fi export_dynamic_flag_spec_CXX='$wl-bexpall' # It seems that -bexpall does not export symbols beginning with # underscore (_), so it is better to generate a list of symbols to # export. always_export_symbols_CXX=yes if test aix,yes = "$with_aix_soname,$aix_use_runtimelinking"; then # Warning - without using the other runtime loading flags (-brtl), # -berok will link without error, but may produce a broken library. # The "-G" linker flag allows undefined symbols. no_undefined_flag_CXX='-bernotok' # Determine the default libpath from the value encoded in an empty # executable. if test set = "${lt_cv_aix_libpath+set}"; then aix_libpath=$lt_cv_aix_libpath else if ${lt_cv_aix_libpath__CXX+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_cxx_try_link "$LINENO"; then : lt_aix_libpath_sed=' /Import File Strings/,/^$/ { /^0/ { s/^0 *\([^ ]*\) *$/\1/ p } }' lt_cv_aix_libpath__CXX=`dump -H conftest$ac_exeext 2>/dev/null | $SED -n -e "$lt_aix_libpath_sed"` # Check for a 64-bit object if we didn't find anything. if test -z "$lt_cv_aix_libpath__CXX"; then lt_cv_aix_libpath__CXX=`dump -HX64 conftest$ac_exeext 2>/dev/null | $SED -n -e "$lt_aix_libpath_sed"` fi fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext if test -z "$lt_cv_aix_libpath__CXX"; then lt_cv_aix_libpath__CXX=/usr/lib:/lib fi fi aix_libpath=$lt_cv_aix_libpath__CXX fi hardcode_libdir_flag_spec_CXX='$wl-blibpath:$libdir:'"$aix_libpath" archive_expsym_cmds_CXX='$CC -o $output_objdir/$soname $libobjs $deplibs $wl'$no_entry_flag' $compiler_flags `if test -n "$allow_undefined_flag"; then func_echo_all "$wl$allow_undefined_flag"; else :; fi` $wl'$exp_sym_flag:\$export_symbols' '$shared_flag else if test ia64 = "$host_cpu"; then hardcode_libdir_flag_spec_CXX='$wl-R $libdir:/usr/lib:/lib' allow_undefined_flag_CXX="-z nodefs" archive_expsym_cmds_CXX="\$CC $shared_flag"' -o $output_objdir/$soname $libobjs $deplibs '"\$wl$no_entry_flag"' $compiler_flags $wl$allow_undefined_flag '"\$wl$exp_sym_flag:\$export_symbols" else # Determine the default libpath from the value encoded in an # empty executable. if test set = "${lt_cv_aix_libpath+set}"; then aix_libpath=$lt_cv_aix_libpath else if ${lt_cv_aix_libpath__CXX+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_cxx_try_link "$LINENO"; then : lt_aix_libpath_sed=' /Import File Strings/,/^$/ { /^0/ { s/^0 *\([^ ]*\) *$/\1/ p } }' lt_cv_aix_libpath__CXX=`dump -H conftest$ac_exeext 2>/dev/null | $SED -n -e "$lt_aix_libpath_sed"` # Check for a 64-bit object if we didn't find anything. if test -z "$lt_cv_aix_libpath__CXX"; then lt_cv_aix_libpath__CXX=`dump -HX64 conftest$ac_exeext 2>/dev/null | $SED -n -e "$lt_aix_libpath_sed"` fi fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext if test -z "$lt_cv_aix_libpath__CXX"; then lt_cv_aix_libpath__CXX=/usr/lib:/lib fi fi aix_libpath=$lt_cv_aix_libpath__CXX fi hardcode_libdir_flag_spec_CXX='$wl-blibpath:$libdir:'"$aix_libpath" # Warning - without using the other run time loading flags, # -berok will link without error, but may produce a broken library. no_undefined_flag_CXX=' $wl-bernotok' allow_undefined_flag_CXX=' $wl-berok' if test yes = "$with_gnu_ld"; then # We only use this code for GNU lds that support --whole-archive. whole_archive_flag_spec_CXX='$wl--whole-archive$convenience $wl--no-whole-archive' else # Exported symbols can be pulled into shared objects from archives whole_archive_flag_spec_CXX='$convenience' fi archive_cmds_need_lc_CXX=yes archive_expsym_cmds_CXX='$RM -r $output_objdir/$realname.d~$MKDIR $output_objdir/$realname.d' # -brtl affects multiple linker settings, -berok does not and is overridden later compiler_flags_filtered='`func_echo_all "$compiler_flags " | $SED -e "s%-brtl\\([, ]\\)%-berok\\1%g"`' if test svr4 != "$with_aix_soname"; then # This is similar to how AIX traditionally builds its shared # libraries. Need -bnortl late, we may have -brtl in LDFLAGS. archive_expsym_cmds_CXX="$archive_expsym_cmds_CXX"'~$CC '$shared_flag_aix' -o $output_objdir/$realname.d/$soname $libobjs $deplibs $wl-bnoentry '$compiler_flags_filtered'$wl-bE:$export_symbols$allow_undefined_flag~$AR $AR_FLAGS $output_objdir/$libname$release.a $output_objdir/$realname.d/$soname' fi if test aix != "$with_aix_soname"; then archive_expsym_cmds_CXX="$archive_expsym_cmds_CXX"'~$CC '$shared_flag_svr4' -o $output_objdir/$realname.d/$shared_archive_member_spec.o $libobjs $deplibs $wl-bnoentry '$compiler_flags_filtered'$wl-bE:$export_symbols$allow_undefined_flag~$STRIP -e $output_objdir/$realname.d/$shared_archive_member_spec.o~( func_echo_all "#! $soname($shared_archive_member_spec.o)"; if test shr_64 = "$shared_archive_member_spec"; then func_echo_all "# 64"; else func_echo_all "# 32"; fi; cat $export_symbols ) > $output_objdir/$realname.d/$shared_archive_member_spec.imp~$AR $AR_FLAGS $output_objdir/$soname $output_objdir/$realname.d/$shared_archive_member_spec.o $output_objdir/$realname.d/$shared_archive_member_spec.imp' else # used by -dlpreopen to get the symbols archive_expsym_cmds_CXX="$archive_expsym_cmds_CXX"'~$MV $output_objdir/$realname.d/$soname $output_objdir' fi archive_expsym_cmds_CXX="$archive_expsym_cmds_CXX"'~$RM -r $output_objdir/$realname.d' fi fi ;; beos*) if $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then allow_undefined_flag_CXX=unsupported # Joseph Beckenbach says some releases of gcc # support --undefined. This deserves some investigation. FIXME archive_cmds_CXX='$CC -nostart $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' else ld_shlibs_CXX=no fi ;; chorus*) case $cc_basename in *) # FIXME: insert proper C++ library support ld_shlibs_CXX=no ;; esac ;; cygwin* | mingw* | pw32* | cegcc*) case $GXX,$cc_basename in ,cl* | no,cl*) # Native MSVC # hardcode_libdir_flag_spec is actually meaningless, as there is # no search path for DLLs. hardcode_libdir_flag_spec_CXX=' ' allow_undefined_flag_CXX=unsupported always_export_symbols_CXX=yes file_list_spec_CXX='@' # Tell ltmain to make .lib files, not .a files. libext=lib # Tell ltmain to make .dll files, not .so files. shrext_cmds=.dll # FIXME: Setting linknames here is a bad hack. archive_cmds_CXX='$CC -o $output_objdir/$soname $libobjs $compiler_flags $deplibs -Wl,-DLL,-IMPLIB:"$tool_output_objdir$libname.dll.lib"~linknames=' archive_expsym_cmds_CXX='if test DEF = "`$SED -n -e '\''s/^[ ]*//'\'' -e '\''/^\(;.*\)*$/d'\'' -e '\''s/^\(EXPORTS\|LIBRARY\)\([ ].*\)*$/DEF/p'\'' -e q $export_symbols`" ; then cp "$export_symbols" "$output_objdir/$soname.def"; echo "$tool_output_objdir$soname.def" > "$output_objdir/$soname.exp"; else $SED -e '\''s/^/-link -EXPORT:/'\'' < $export_symbols > $output_objdir/$soname.exp; fi~ $CC -o $tool_output_objdir$soname $libobjs $compiler_flags $deplibs "@$tool_output_objdir$soname.exp" -Wl,-DLL,-IMPLIB:"$tool_output_objdir$libname.dll.lib"~ linknames=' # The linker will not automatically build a static lib if we build a DLL. # _LT_TAGVAR(old_archive_from_new_cmds, CXX)='true' enable_shared_with_static_runtimes_CXX=yes # Don't use ranlib old_postinstall_cmds_CXX='chmod 644 $oldlib' postlink_cmds_CXX='lt_outputfile="@OUTPUT@"~ lt_tool_outputfile="@TOOL_OUTPUT@"~ case $lt_outputfile in *.exe|*.EXE) ;; *) lt_outputfile=$lt_outputfile.exe lt_tool_outputfile=$lt_tool_outputfile.exe ;; esac~ func_to_tool_file "$lt_outputfile"~ if test : != "$MANIFEST_TOOL" && test -f "$lt_outputfile.manifest"; then $MANIFEST_TOOL -manifest "$lt_tool_outputfile.manifest" -outputresource:"$lt_tool_outputfile" || exit 1; $RM "$lt_outputfile.manifest"; fi' ;; *) # g++ # _LT_TAGVAR(hardcode_libdir_flag_spec, CXX) is actually meaningless, # as there is no search path for DLLs. hardcode_libdir_flag_spec_CXX='-L$libdir' export_dynamic_flag_spec_CXX='$wl--export-all-symbols' allow_undefined_flag_CXX=unsupported always_export_symbols_CXX=no enable_shared_with_static_runtimes_CXX=yes if $LD --help 2>&1 | $GREP 'auto-import' > /dev/null; then archive_cmds_CXX='$CC -shared -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags -o $output_objdir/$soname $wl--enable-auto-image-base -Xlinker --out-implib -Xlinker $lib' # If the export-symbols file already is a .def file, use it as # is; otherwise, prepend EXPORTS... archive_expsym_cmds_CXX='if test DEF = "`$SED -n -e '\''s/^[ ]*//'\'' -e '\''/^\(;.*\)*$/d'\'' -e '\''s/^\(EXPORTS\|LIBRARY\)\([ ].*\)*$/DEF/p'\'' -e q $export_symbols`" ; then cp $export_symbols $output_objdir/$soname.def; else echo EXPORTS > $output_objdir/$soname.def; cat $export_symbols >> $output_objdir/$soname.def; fi~ $CC -shared -nostdlib $output_objdir/$soname.def $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags -o $output_objdir/$soname $wl--enable-auto-image-base -Xlinker --out-implib -Xlinker $lib' else ld_shlibs_CXX=no fi ;; esac ;; darwin* | rhapsody*) archive_cmds_need_lc_CXX=no hardcode_direct_CXX=no hardcode_automatic_CXX=yes hardcode_shlibpath_var_CXX=unsupported if test yes = "$lt_cv_ld_force_load"; then whole_archive_flag_spec_CXX='`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience $wl-force_load,$conv\"; done; func_echo_all \"$new_convenience\"`' else whole_archive_flag_spec_CXX='' fi link_all_deplibs_CXX=yes allow_undefined_flag_CXX=$_lt_dar_allow_undefined case $cc_basename in ifort*|nagfor*) _lt_dar_can_shared=yes ;; *) _lt_dar_can_shared=$GCC ;; esac if test yes = "$_lt_dar_can_shared"; then output_verbose_link_cmd=func_echo_all archive_cmds_CXX="\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$libobjs \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring $_lt_dar_single_mod$_lt_dsymutil" module_cmds_CXX="\$CC \$allow_undefined_flag -o \$lib -bundle \$libobjs \$deplibs \$compiler_flags$_lt_dsymutil" archive_expsym_cmds_CXX="sed 's|^|_|' < \$export_symbols > \$output_objdir/\$libname-symbols.expsym~\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$libobjs \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring $_lt_dar_single_mod$_lt_dar_export_syms$_lt_dsymutil" module_expsym_cmds_CXX="sed -e 's|^|_|' < \$export_symbols > \$output_objdir/\$libname-symbols.expsym~\$CC \$allow_undefined_flag -o \$lib -bundle \$libobjs \$deplibs \$compiler_flags$_lt_dar_export_syms$_lt_dsymutil" if test yes != "$lt_cv_apple_cc_single_mod"; then archive_cmds_CXX="\$CC -r -keep_private_externs -nostdlib -o \$lib-master.o \$libobjs~\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$lib-master.o \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring$_lt_dsymutil" archive_expsym_cmds_CXX="sed 's|^|_|' < \$export_symbols > \$output_objdir/\$libname-symbols.expsym~\$CC -r -keep_private_externs -nostdlib -o \$lib-master.o \$libobjs~\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$lib-master.o \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring$_lt_dar_export_syms$_lt_dsymutil" fi else ld_shlibs_CXX=no fi ;; os2*) hardcode_libdir_flag_spec_CXX='-L$libdir' hardcode_minus_L_CXX=yes allow_undefined_flag_CXX=unsupported shrext_cmds=.dll archive_cmds_CXX='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ $ECHO EXPORTS >> $output_objdir/$libname.def~ emxexp $libobjs | $SED /"_DLL_InitTerm"/d >> $output_objdir/$libname.def~ $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ emximp -o $lib $output_objdir/$libname.def' archive_expsym_cmds_CXX='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ $ECHO EXPORTS >> $output_objdir/$libname.def~ prefix_cmds="$SED"~ if test EXPORTS = "`$SED 1q $export_symbols`"; then prefix_cmds="$prefix_cmds -e 1d"; fi~ prefix_cmds="$prefix_cmds -e \"s/^\(.*\)$/_\1/g\""~ cat $export_symbols | $prefix_cmds >> $output_objdir/$libname.def~ $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ emximp -o $lib $output_objdir/$libname.def' old_archive_From_new_cmds_CXX='emximp -o $output_objdir/${libname}_dll.a $output_objdir/$libname.def' enable_shared_with_static_runtimes_CXX=yes ;; dgux*) case $cc_basename in ec++*) # FIXME: insert proper C++ library support ld_shlibs_CXX=no ;; ghcx*) # Green Hills C++ Compiler # FIXME: insert proper C++ library support ld_shlibs_CXX=no ;; *) # FIXME: insert proper C++ library support ld_shlibs_CXX=no ;; esac ;; freebsd2.*) # C++ shared libraries reported to be fairly broken before # switch to ELF ld_shlibs_CXX=no ;; freebsd-elf*) archive_cmds_need_lc_CXX=no ;; freebsd* | dragonfly*) # FreeBSD 3 and later use GNU C++ and GNU ld with standard ELF # conventions ld_shlibs_CXX=yes ;; haiku*) archive_cmds_CXX='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' link_all_deplibs_CXX=yes ;; hpux9*) hardcode_libdir_flag_spec_CXX='$wl+b $wl$libdir' hardcode_libdir_separator_CXX=: export_dynamic_flag_spec_CXX='$wl-E' hardcode_direct_CXX=yes hardcode_minus_L_CXX=yes # Not in the search PATH, # but as the default # location of the library. case $cc_basename in CC*) # FIXME: insert proper C++ library support ld_shlibs_CXX=no ;; aCC*) archive_cmds_CXX='$RM $output_objdir/$soname~$CC -b $wl+b $wl$install_libdir -o $output_objdir/$soname $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags~test "x$output_objdir/$soname" = "x$lib" || mv $output_objdir/$soname $lib' # Commands to make compiler produce verbose output that lists # what "hidden" libraries, object files and flags are used when # linking a shared library. # # There doesn't appear to be a way to prevent this compiler from # explicitly linking system object files so we need to strip them # from the output so that they don't get included in the library # dependencies. output_verbose_link_cmd='templist=`($CC -b $CFLAGS -v conftest.$objext 2>&1) | $EGREP "\-L"`; list= ; for z in $templist; do case $z in conftest.$objext) list="$list $z";; *.$objext);; *) list="$list $z";;esac; done; func_echo_all "$list"' ;; *) if test yes = "$GXX"; then archive_cmds_CXX='$RM $output_objdir/$soname~$CC -shared -nostdlib $pic_flag $wl+b $wl$install_libdir -o $output_objdir/$soname $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags~test "x$output_objdir/$soname" = "x$lib" || mv $output_objdir/$soname $lib' else # FIXME: insert proper C++ library support ld_shlibs_CXX=no fi ;; esac ;; hpux10*|hpux11*) if test no = "$with_gnu_ld"; then hardcode_libdir_flag_spec_CXX='$wl+b $wl$libdir' hardcode_libdir_separator_CXX=: case $host_cpu in hppa*64*|ia64*) ;; *) export_dynamic_flag_spec_CXX='$wl-E' ;; esac fi case $host_cpu in hppa*64*|ia64*) hardcode_direct_CXX=no hardcode_shlibpath_var_CXX=no ;; *) hardcode_direct_CXX=yes hardcode_direct_absolute_CXX=yes hardcode_minus_L_CXX=yes # Not in the search PATH, # but as the default # location of the library. ;; esac case $cc_basename in CC*) # FIXME: insert proper C++ library support ld_shlibs_CXX=no ;; aCC*) case $host_cpu in hppa*64*) archive_cmds_CXX='$CC -b $wl+h $wl$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' ;; ia64*) archive_cmds_CXX='$CC -b $wl+h $wl$soname $wl+nodefaultrpath -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' ;; *) archive_cmds_CXX='$CC -b $wl+h $wl$soname $wl+b $wl$install_libdir -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' ;; esac # Commands to make compiler produce verbose output that lists # what "hidden" libraries, object files and flags are used when # linking a shared library. # # There doesn't appear to be a way to prevent this compiler from # explicitly linking system object files so we need to strip them # from the output so that they don't get included in the library # dependencies. output_verbose_link_cmd='templist=`($CC -b $CFLAGS -v conftest.$objext 2>&1) | $GREP "\-L"`; list= ; for z in $templist; do case $z in conftest.$objext) list="$list $z";; *.$objext);; *) list="$list $z";;esac; done; func_echo_all "$list"' ;; *) if test yes = "$GXX"; then if test no = "$with_gnu_ld"; then case $host_cpu in hppa*64*) archive_cmds_CXX='$CC -shared -nostdlib -fPIC $wl+h $wl$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' ;; ia64*) archive_cmds_CXX='$CC -shared -nostdlib $pic_flag $wl+h $wl$soname $wl+nodefaultrpath -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' ;; *) archive_cmds_CXX='$CC -shared -nostdlib $pic_flag $wl+h $wl$soname $wl+b $wl$install_libdir -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' ;; esac fi else # FIXME: insert proper C++ library support ld_shlibs_CXX=no fi ;; esac ;; interix[3-9]*) hardcode_direct_CXX=no hardcode_shlibpath_var_CXX=no hardcode_libdir_flag_spec_CXX='$wl-rpath,$libdir' export_dynamic_flag_spec_CXX='$wl-E' # Hack: On Interix 3.x, we cannot compile PIC because of a broken gcc. # Instead, shared libraries are loaded at an image base (0x10000000 by # default) and relocated if they conflict, which is a slow very memory # consuming and fragmenting process. To avoid this, we pick a random, # 256 KiB-aligned image base between 0x50000000 and 0x6FFC0000 at link # time. Moving up from 0x10000000 also allows more sbrk(2) space. archive_cmds_CXX='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-h,$soname $wl--image-base,`expr ${RANDOM-$$} % 4096 / 2 \* 262144 + 1342177280` -o $lib' archive_expsym_cmds_CXX='sed "s|^|_|" $export_symbols >$output_objdir/$soname.expsym~$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-h,$soname $wl--retain-symbols-file,$output_objdir/$soname.expsym $wl--image-base,`expr ${RANDOM-$$} % 4096 / 2 \* 262144 + 1342177280` -o $lib' ;; irix5* | irix6*) case $cc_basename in CC*) # SGI C++ archive_cmds_CXX='$CC -shared -all -multigot $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib' # Archives containing C++ object files must be created using # "CC -ar", where "CC" is the IRIX C++ compiler. This is # necessary to make sure instantiated templates are included # in the archive. old_archive_cmds_CXX='$CC -ar -WR,-u -o $oldlib $oldobjs' ;; *) if test yes = "$GXX"; then if test no = "$with_gnu_ld"; then archive_cmds_CXX='$CC -shared $pic_flag -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations -o $lib' else archive_cmds_CXX='$CC -shared $pic_flag -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` -o $lib' fi fi link_all_deplibs_CXX=yes ;; esac hardcode_libdir_flag_spec_CXX='$wl-rpath $wl$libdir' hardcode_libdir_separator_CXX=: inherit_rpath_CXX=yes ;; linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*) case $cc_basename in KCC*) # Kuck and Associates, Inc. (KAI) C++ Compiler # KCC will only create a shared library if the output file # ends with ".so" (or ".sl" for HP-UX), so rename the library # to its proper name (with version) after linking. archive_cmds_CXX='tempext=`echo $shared_ext | $SED -e '\''s/\([^()0-9A-Za-z{}]\)/\\\\\1/g'\''`; templib=`echo $lib | $SED -e "s/\$tempext\..*/.so/"`; $CC $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags --soname $soname -o \$templib; mv \$templib $lib' archive_expsym_cmds_CXX='tempext=`echo $shared_ext | $SED -e '\''s/\([^()0-9A-Za-z{}]\)/\\\\\1/g'\''`; templib=`echo $lib | $SED -e "s/\$tempext\..*/.so/"`; $CC $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags --soname $soname -o \$templib $wl-retain-symbols-file,$export_symbols; mv \$templib $lib' # Commands to make compiler produce verbose output that lists # what "hidden" libraries, object files and flags are used when # linking a shared library. # # There doesn't appear to be a way to prevent this compiler from # explicitly linking system object files so we need to strip them # from the output so that they don't get included in the library # dependencies. output_verbose_link_cmd='templist=`$CC $CFLAGS -v conftest.$objext -o libconftest$shared_ext 2>&1 | $GREP "ld"`; rm -f libconftest$shared_ext; list= ; for z in $templist; do case $z in conftest.$objext) list="$list $z";; *.$objext);; *) list="$list $z";;esac; done; func_echo_all "$list"' hardcode_libdir_flag_spec_CXX='$wl-rpath,$libdir' export_dynamic_flag_spec_CXX='$wl--export-dynamic' # Archives containing C++ object files must be created using # "CC -Bstatic", where "CC" is the KAI C++ compiler. old_archive_cmds_CXX='$CC -Bstatic -o $oldlib $oldobjs' ;; icpc* | ecpc* ) # Intel C++ with_gnu_ld=yes # version 8.0 and above of icpc choke on multiply defined symbols # if we add $predep_objects and $postdep_objects, however 7.1 and # earlier do not add the objects themselves. case `$CC -V 2>&1` in *"Version 7."*) archive_cmds_CXX='$CC -shared $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname -o $lib' archive_expsym_cmds_CXX='$CC -shared $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' ;; *) # Version 8.0 or newer tmp_idyn= case $host_cpu in ia64*) tmp_idyn=' -i_dynamic';; esac archive_cmds_CXX='$CC -shared'"$tmp_idyn"' $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' archive_expsym_cmds_CXX='$CC -shared'"$tmp_idyn"' $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' ;; esac archive_cmds_need_lc_CXX=no hardcode_libdir_flag_spec_CXX='$wl-rpath,$libdir' export_dynamic_flag_spec_CXX='$wl--export-dynamic' whole_archive_flag_spec_CXX='$wl--whole-archive$convenience $wl--no-whole-archive' ;; pgCC* | pgcpp*) # Portland Group C++ compiler case `$CC -V` in *pgCC\ [1-5].* | *pgcpp\ [1-5].*) prelink_cmds_CXX='tpldir=Template.dir~ rm -rf $tpldir~ $CC --prelink_objects --instantiation_dir $tpldir $objs $libobjs $compile_deplibs~ compile_command="$compile_command `find $tpldir -name \*.o | sort | $NL2SP`"' old_archive_cmds_CXX='tpldir=Template.dir~ rm -rf $tpldir~ $CC --prelink_objects --instantiation_dir $tpldir $oldobjs$old_deplibs~ $AR $AR_FLAGS $oldlib$oldobjs$old_deplibs `find $tpldir -name \*.o | sort | $NL2SP`~ $RANLIB $oldlib' archive_cmds_CXX='tpldir=Template.dir~ rm -rf $tpldir~ $CC --prelink_objects --instantiation_dir $tpldir $predep_objects $libobjs $deplibs $convenience $postdep_objects~ $CC -shared $pic_flag $predep_objects $libobjs $deplibs `find $tpldir -name \*.o | sort | $NL2SP` $postdep_objects $compiler_flags $wl-soname $wl$soname -o $lib' archive_expsym_cmds_CXX='tpldir=Template.dir~ rm -rf $tpldir~ $CC --prelink_objects --instantiation_dir $tpldir $predep_objects $libobjs $deplibs $convenience $postdep_objects~ $CC -shared $pic_flag $predep_objects $libobjs $deplibs `find $tpldir -name \*.o | sort | $NL2SP` $postdep_objects $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' ;; *) # Version 6 and above use weak symbols archive_cmds_CXX='$CC -shared $pic_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname -o $lib' archive_expsym_cmds_CXX='$CC -shared $pic_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' ;; esac hardcode_libdir_flag_spec_CXX='$wl--rpath $wl$libdir' export_dynamic_flag_spec_CXX='$wl--export-dynamic' whole_archive_flag_spec_CXX='$wl--whole-archive`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' ;; cxx*) # Compaq C++ archive_cmds_CXX='$CC -shared $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname -o $lib' archive_expsym_cmds_CXX='$CC -shared $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname -o $lib $wl-retain-symbols-file $wl$export_symbols' runpath_var=LD_RUN_PATH hardcode_libdir_flag_spec_CXX='-rpath $libdir' hardcode_libdir_separator_CXX=: # Commands to make compiler produce verbose output that lists # what "hidden" libraries, object files and flags are used when # linking a shared library. # # There doesn't appear to be a way to prevent this compiler from # explicitly linking system object files so we need to strip them # from the output so that they don't get included in the library # dependencies. output_verbose_link_cmd='templist=`$CC -shared $CFLAGS -v conftest.$objext 2>&1 | $GREP "ld"`; templist=`func_echo_all "$templist" | $SED "s/\(^.*ld.*\)\( .*ld .*$\)/\1/"`; list= ; for z in $templist; do case $z in conftest.$objext) list="$list $z";; *.$objext);; *) list="$list $z";;esac; done; func_echo_all "X$list" | $Xsed' ;; xl* | mpixl* | bgxl*) # IBM XL 8.0 on PPC, with GNU ld hardcode_libdir_flag_spec_CXX='$wl-rpath $wl$libdir' export_dynamic_flag_spec_CXX='$wl--export-dynamic' archive_cmds_CXX='$CC -qmkshrobj $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' if test yes = "$supports_anon_versioning"; then archive_expsym_cmds_CXX='echo "{ global:" > $output_objdir/$libname.ver~ cat $export_symbols | sed -e "s/\(.*\)/\1;/" >> $output_objdir/$libname.ver~ echo "local: *; };" >> $output_objdir/$libname.ver~ $CC -qmkshrobj $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-version-script $wl$output_objdir/$libname.ver -o $lib' fi ;; *) case `$CC -V 2>&1 | sed 5q` in *Sun\ C*) # Sun C++ 5.9 no_undefined_flag_CXX=' -zdefs' archive_cmds_CXX='$CC -G$allow_undefined_flag -h$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' archive_expsym_cmds_CXX='$CC -G$allow_undefined_flag -h$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-retain-symbols-file $wl$export_symbols' hardcode_libdir_flag_spec_CXX='-R$libdir' whole_archive_flag_spec_CXX='$wl--whole-archive`new_convenience=; for conv in $convenience\"\"; do test -z \"$conv\" || new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' compiler_needs_object_CXX=yes # Not sure whether something based on # $CC $CFLAGS -v conftest.$objext -o libconftest$shared_ext 2>&1 # would be better. output_verbose_link_cmd='func_echo_all' # Archives containing C++ object files must be created using # "CC -xar", where "CC" is the Sun C++ compiler. This is # necessary to make sure instantiated templates are included # in the archive. old_archive_cmds_CXX='$CC -xar -o $oldlib $oldobjs' ;; esac ;; esac ;; lynxos*) # FIXME: insert proper C++ library support ld_shlibs_CXX=no ;; m88k*) # FIXME: insert proper C++ library support ld_shlibs_CXX=no ;; mvs*) case $cc_basename in cxx*) # FIXME: insert proper C++ library support ld_shlibs_CXX=no ;; *) # FIXME: insert proper C++ library support ld_shlibs_CXX=no ;; esac ;; netbsd*) if echo __ELF__ | $CC -E - | $GREP __ELF__ >/dev/null; then archive_cmds_CXX='$LD -Bshareable -o $lib $predep_objects $libobjs $deplibs $postdep_objects $linker_flags' wlarc= hardcode_libdir_flag_spec_CXX='-R$libdir' hardcode_direct_CXX=yes hardcode_shlibpath_var_CXX=no fi # Workaround some broken pre-1.5 toolchains output_verbose_link_cmd='$CC -shared $CFLAGS -v conftest.$objext 2>&1 | $GREP conftest.$objext | $SED -e "s:-lgcc -lc -lgcc::"' ;; *nto* | *qnx*) ld_shlibs_CXX=yes ;; openbsd* | bitrig*) if test -f /usr/libexec/ld.so; then hardcode_direct_CXX=yes hardcode_shlibpath_var_CXX=no hardcode_direct_absolute_CXX=yes archive_cmds_CXX='$CC -shared $pic_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags -o $lib' hardcode_libdir_flag_spec_CXX='$wl-rpath,$libdir' if test -z "`echo __ELF__ | $CC -E - | grep __ELF__`"; then archive_expsym_cmds_CXX='$CC -shared $pic_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-retain-symbols-file,$export_symbols -o $lib' export_dynamic_flag_spec_CXX='$wl-E' whole_archive_flag_spec_CXX=$wlarc'--whole-archive$convenience '$wlarc'--no-whole-archive' fi output_verbose_link_cmd=func_echo_all else ld_shlibs_CXX=no fi ;; osf3* | osf4* | osf5*) case $cc_basename in KCC*) # Kuck and Associates, Inc. (KAI) C++ Compiler # KCC will only create a shared library if the output file # ends with ".so" (or ".sl" for HP-UX), so rename the library # to its proper name (with version) after linking. archive_cmds_CXX='tempext=`echo $shared_ext | $SED -e '\''s/\([^()0-9A-Za-z{}]\)/\\\\\1/g'\''`; templib=`echo "$lib" | $SED -e "s/\$tempext\..*/.so/"`; $CC $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags --soname $soname -o \$templib; mv \$templib $lib' hardcode_libdir_flag_spec_CXX='$wl-rpath,$libdir' hardcode_libdir_separator_CXX=: # Archives containing C++ object files must be created using # the KAI C++ compiler. case $host in osf3*) old_archive_cmds_CXX='$CC -Bstatic -o $oldlib $oldobjs' ;; *) old_archive_cmds_CXX='$CC -o $oldlib $oldobjs' ;; esac ;; RCC*) # Rational C++ 2.4.1 # FIXME: insert proper C++ library support ld_shlibs_CXX=no ;; cxx*) case $host in osf3*) allow_undefined_flag_CXX=' $wl-expect_unresolved $wl\*' archive_cmds_CXX='$CC -shared$allow_undefined_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $soname `test -n "$verstring" && func_echo_all "$wl-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib' hardcode_libdir_flag_spec_CXX='$wl-rpath $wl$libdir' ;; *) allow_undefined_flag_CXX=' -expect_unresolved \*' archive_cmds_CXX='$CC -shared$allow_undefined_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags -msym -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib' archive_expsym_cmds_CXX='for i in `cat $export_symbols`; do printf "%s %s\\n" -exported_symbol "\$i" >> $lib.exp; done~ echo "-hidden">> $lib.exp~ $CC -shared$allow_undefined_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags -msym -soname $soname $wl-input $wl$lib.exp `test -n "$verstring" && $ECHO "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib~ $RM $lib.exp' hardcode_libdir_flag_spec_CXX='-rpath $libdir' ;; esac hardcode_libdir_separator_CXX=: # Commands to make compiler produce verbose output that lists # what "hidden" libraries, object files and flags are used when # linking a shared library. # # There doesn't appear to be a way to prevent this compiler from # explicitly linking system object files so we need to strip them # from the output so that they don't get included in the library # dependencies. output_verbose_link_cmd='templist=`$CC -shared $CFLAGS -v conftest.$objext 2>&1 | $GREP "ld" | $GREP -v "ld:"`; templist=`func_echo_all "$templist" | $SED "s/\(^.*ld.*\)\( .*ld.*$\)/\1/"`; list= ; for z in $templist; do case $z in conftest.$objext) list="$list $z";; *.$objext);; *) list="$list $z";;esac; done; func_echo_all "$list"' ;; *) if test yes,no = "$GXX,$with_gnu_ld"; then allow_undefined_flag_CXX=' $wl-expect_unresolved $wl\*' case $host in osf3*) archive_cmds_CXX='$CC -shared -nostdlib $allow_undefined_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations -o $lib' ;; *) archive_cmds_CXX='$CC -shared $pic_flag -nostdlib $allow_undefined_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-msym $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations -o $lib' ;; esac hardcode_libdir_flag_spec_CXX='$wl-rpath $wl$libdir' hardcode_libdir_separator_CXX=: # Commands to make compiler produce verbose output that lists # what "hidden" libraries, object files and flags are used when # linking a shared library. output_verbose_link_cmd='$CC -shared $CFLAGS -v conftest.$objext 2>&1 | $GREP -v "^Configured with:" | $GREP "\-L"' else # FIXME: insert proper C++ library support ld_shlibs_CXX=no fi ;; esac ;; psos*) # FIXME: insert proper C++ library support ld_shlibs_CXX=no ;; sunos4*) case $cc_basename in CC*) # Sun C++ 4.x # FIXME: insert proper C++ library support ld_shlibs_CXX=no ;; lcc*) # Lucid # FIXME: insert proper C++ library support ld_shlibs_CXX=no ;; *) # FIXME: insert proper C++ library support ld_shlibs_CXX=no ;; esac ;; solaris*) case $cc_basename in CC* | sunCC*) # Sun C++ 4.2, 5.x and Centerline C++ archive_cmds_need_lc_CXX=yes no_undefined_flag_CXX=' -zdefs' archive_cmds_CXX='$CC -G$allow_undefined_flag -h$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' archive_expsym_cmds_CXX='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ $CC -G$allow_undefined_flag $wl-M $wl$lib.exp -h$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags~$RM $lib.exp' hardcode_libdir_flag_spec_CXX='-R$libdir' hardcode_shlibpath_var_CXX=no case $host_os in solaris2.[0-5] | solaris2.[0-5].*) ;; *) # The compiler driver will combine and reorder linker options, # but understands '-z linker_flag'. # Supported since Solaris 2.6 (maybe 2.5.1?) whole_archive_flag_spec_CXX='-z allextract$convenience -z defaultextract' ;; esac link_all_deplibs_CXX=yes output_verbose_link_cmd='func_echo_all' # Archives containing C++ object files must be created using # "CC -xar", where "CC" is the Sun C++ compiler. This is # necessary to make sure instantiated templates are included # in the archive. old_archive_cmds_CXX='$CC -xar -o $oldlib $oldobjs' ;; gcx*) # Green Hills C++ Compiler archive_cmds_CXX='$CC -shared $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-h $wl$soname -o $lib' # The C++ compiler must be used to create the archive. old_archive_cmds_CXX='$CC $LDFLAGS -archive -o $oldlib $oldobjs' ;; *) # GNU C++ compiler with Solaris linker if test yes,no = "$GXX,$with_gnu_ld"; then no_undefined_flag_CXX=' $wl-z ${wl}defs' if $CC --version | $GREP -v '^2\.7' > /dev/null; then archive_cmds_CXX='$CC -shared $pic_flag -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-h $wl$soname -o $lib' archive_expsym_cmds_CXX='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ $CC -shared $pic_flag -nostdlib $wl-M $wl$lib.exp $wl-h $wl$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags~$RM $lib.exp' # Commands to make compiler produce verbose output that lists # what "hidden" libraries, object files and flags are used when # linking a shared library. output_verbose_link_cmd='$CC -shared $CFLAGS -v conftest.$objext 2>&1 | $GREP -v "^Configured with:" | $GREP "\-L"' else # g++ 2.7 appears to require '-G' NOT '-shared' on this # platform. archive_cmds_CXX='$CC -G -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-h $wl$soname -o $lib' archive_expsym_cmds_CXX='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ $CC -G -nostdlib $wl-M $wl$lib.exp $wl-h $wl$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags~$RM $lib.exp' # Commands to make compiler produce verbose output that lists # what "hidden" libraries, object files and flags are used when # linking a shared library. output_verbose_link_cmd='$CC -G $CFLAGS -v conftest.$objext 2>&1 | $GREP -v "^Configured with:" | $GREP "\-L"' fi hardcode_libdir_flag_spec_CXX='$wl-R $wl$libdir' case $host_os in solaris2.[0-5] | solaris2.[0-5].*) ;; *) whole_archive_flag_spec_CXX='$wl-z ${wl}allextract$convenience $wl-z ${wl}defaultextract' ;; esac fi ;; esac ;; sysv4*uw2* | sysv5OpenUNIX* | sysv5UnixWare7.[01].[10]* | unixware7* | sco3.2v5.0.[024]*) no_undefined_flag_CXX='$wl-z,text' archive_cmds_need_lc_CXX=no hardcode_shlibpath_var_CXX=no runpath_var='LD_RUN_PATH' case $cc_basename in CC*) archive_cmds_CXX='$CC -G $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' archive_expsym_cmds_CXX='$CC -G $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' ;; *) archive_cmds_CXX='$CC -shared $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' archive_expsym_cmds_CXX='$CC -shared $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' ;; esac ;; sysv5* | sco3.2v5* | sco5v6*) # Note: We CANNOT use -z defs as we might desire, because we do not # link with -lc, and that would cause any symbols used from libc to # always be unresolved, which means just about no library would # ever link correctly. If we're not using GNU ld we use -z text # though, which does catch some bad symbols but isn't as heavy-handed # as -z defs. no_undefined_flag_CXX='$wl-z,text' allow_undefined_flag_CXX='$wl-z,nodefs' archive_cmds_need_lc_CXX=no hardcode_shlibpath_var_CXX=no hardcode_libdir_flag_spec_CXX='$wl-R,$libdir' hardcode_libdir_separator_CXX=':' link_all_deplibs_CXX=yes export_dynamic_flag_spec_CXX='$wl-Bexport' runpath_var='LD_RUN_PATH' case $cc_basename in CC*) archive_cmds_CXX='$CC -G $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' archive_expsym_cmds_CXX='$CC -G $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' old_archive_cmds_CXX='$CC -Tprelink_objects $oldobjs~ '"$old_archive_cmds_CXX" reload_cmds_CXX='$CC -Tprelink_objects $reload_objs~ '"$reload_cmds_CXX" ;; *) archive_cmds_CXX='$CC -shared $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' archive_expsym_cmds_CXX='$CC -shared $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' ;; esac ;; tandem*) case $cc_basename in NCC*) # NonStop-UX NCC 3.20 # FIXME: insert proper C++ library support ld_shlibs_CXX=no ;; *) # FIXME: insert proper C++ library support ld_shlibs_CXX=no ;; esac ;; vxworks*) # FIXME: insert proper C++ library support ld_shlibs_CXX=no ;; *) # FIXME: insert proper C++ library support ld_shlibs_CXX=no ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ld_shlibs_CXX" >&5 $as_echo "$ld_shlibs_CXX" >&6; } test no = "$ld_shlibs_CXX" && can_build_shared=no GCC_CXX=$GXX LD_CXX=$LD ## CAVEAT EMPTOR: ## There is no encapsulation within the following macros, do not change ## the running order or otherwise move them around unless you know exactly ## what you are doing... # Dependencies to place before and after the object being linked: predep_objects_CXX= postdep_objects_CXX= predeps_CXX= postdeps_CXX= compiler_lib_search_path_CXX= cat > conftest.$ac_ext <<_LT_EOF class Foo { public: Foo (void) { a = 0; } private: int a; }; _LT_EOF _lt_libdeps_save_CFLAGS=$CFLAGS case "$CC $CFLAGS " in #( *\ -flto*\ *) CFLAGS="$CFLAGS -fno-lto" ;; *\ -fwhopr*\ *) CFLAGS="$CFLAGS -fno-whopr" ;; *\ -fuse-linker-plugin*\ *) CFLAGS="$CFLAGS -fno-use-linker-plugin" ;; esac if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 (eval $ac_compile) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then # Parse the compiler output and extract the necessary # objects, libraries and library flags. # Sentinel used to keep track of whether or not we are before # the conftest object file. pre_test_object_deps_done=no for p in `eval "$output_verbose_link_cmd"`; do case $prev$p in -L* | -R* | -l*) # Some compilers place space between "-{L,R}" and the path. # Remove the space. if test x-L = "$p" || test x-R = "$p"; then prev=$p continue fi # Expand the sysroot to ease extracting the directories later. if test -z "$prev"; then case $p in -L*) func_stripname_cnf '-L' '' "$p"; prev=-L; p=$func_stripname_result ;; -R*) func_stripname_cnf '-R' '' "$p"; prev=-R; p=$func_stripname_result ;; -l*) func_stripname_cnf '-l' '' "$p"; prev=-l; p=$func_stripname_result ;; esac fi case $p in =*) func_stripname_cnf '=' '' "$p"; p=$lt_sysroot$func_stripname_result ;; esac if test no = "$pre_test_object_deps_done"; then case $prev in -L | -R) # Internal compiler library paths should come after those # provided the user. The postdeps already come after the # user supplied libs so there is no need to process them. if test -z "$compiler_lib_search_path_CXX"; then compiler_lib_search_path_CXX=$prev$p else compiler_lib_search_path_CXX="${compiler_lib_search_path_CXX} $prev$p" fi ;; # The "-l" case would never come before the object being # linked, so don't bother handling this case. esac else if test -z "$postdeps_CXX"; then postdeps_CXX=$prev$p else postdeps_CXX="${postdeps_CXX} $prev$p" fi fi prev= ;; *.lto.$objext) ;; # Ignore GCC LTO objects *.$objext) # This assumes that the test object file only shows up # once in the compiler output. if test "$p" = "conftest.$objext"; then pre_test_object_deps_done=yes continue fi if test no = "$pre_test_object_deps_done"; then if test -z "$predep_objects_CXX"; then predep_objects_CXX=$p else predep_objects_CXX="$predep_objects_CXX $p" fi else if test -z "$postdep_objects_CXX"; then postdep_objects_CXX=$p else postdep_objects_CXX="$postdep_objects_CXX $p" fi fi ;; *) ;; # Ignore the rest. esac done # Clean up. rm -f a.out a.exe else echo "libtool.m4: error: problem compiling CXX test program" fi $RM -f confest.$objext CFLAGS=$_lt_libdeps_save_CFLAGS # PORTME: override above test on systems where it is broken case $host_os in interix[3-9]*) # Interix 3.5 installs completely hosed .la files for C++, so rather than # hack all around it, let's just trust "g++" to DTRT. predep_objects_CXX= postdep_objects_CXX= postdeps_CXX= ;; esac case " $postdeps_CXX " in *" -lc "*) archive_cmds_need_lc_CXX=no ;; esac compiler_lib_search_dirs_CXX= if test -n "${compiler_lib_search_path_CXX}"; then compiler_lib_search_dirs_CXX=`echo " ${compiler_lib_search_path_CXX}" | $SED -e 's! -L! !g' -e 's!^ !!'` fi lt_prog_compiler_wl_CXX= lt_prog_compiler_pic_CXX= lt_prog_compiler_static_CXX= # C++ specific cases for pic, static, wl, etc. if test yes = "$GXX"; then lt_prog_compiler_wl_CXX='-Wl,' lt_prog_compiler_static_CXX='-static' case $host_os in aix*) # All AIX code is PIC. if test ia64 = "$host_cpu"; then # AIX 5 now supports IA64 processor lt_prog_compiler_static_CXX='-Bstatic' fi lt_prog_compiler_pic_CXX='-fPIC' ;; amigaos*) case $host_cpu in powerpc) # see comment about AmigaOS4 .so support lt_prog_compiler_pic_CXX='-fPIC' ;; m68k) # FIXME: we need at least 68020 code to build shared libraries, but # adding the '-m68020' flag to GCC prevents building anything better, # like '-m68040'. lt_prog_compiler_pic_CXX='-m68020 -resident32 -malways-restore-a4' ;; esac ;; beos* | irix5* | irix6* | nonstopux* | osf3* | osf4* | osf5*) # PIC is the default for these OSes. ;; mingw* | cygwin* | os2* | pw32* | cegcc*) # This hack is so that the source file can tell whether it is being # built for inclusion in a dll (and should export symbols for example). # Although the cygwin gcc ignores -fPIC, still need this for old-style # (--disable-auto-import) libraries lt_prog_compiler_pic_CXX='-DDLL_EXPORT' case $host_os in os2*) lt_prog_compiler_static_CXX='$wl-static' ;; esac ;; darwin* | rhapsody*) # PIC is the default on this platform # Common symbols not allowed in MH_DYLIB files lt_prog_compiler_pic_CXX='-fno-common' ;; *djgpp*) # DJGPP does not support shared libraries at all lt_prog_compiler_pic_CXX= ;; haiku*) # PIC is the default for Haiku. # The "-static" flag exists, but is broken. lt_prog_compiler_static_CXX= ;; interix[3-9]*) # Interix 3.x gcc -fpic/-fPIC options generate broken code. # Instead, we relocate shared libraries at runtime. ;; sysv4*MP*) if test -d /usr/nec; then lt_prog_compiler_pic_CXX=-Kconform_pic fi ;; hpux*) # PIC is the default for 64-bit PA HP-UX, but not for 32-bit # PA HP-UX. On IA64 HP-UX, PIC is the default but the pic flag # sets the default TLS model and affects inlining. case $host_cpu in hppa*64*) ;; *) lt_prog_compiler_pic_CXX='-fPIC' ;; esac ;; *qnx* | *nto*) # QNX uses GNU C++, but need to define -shared option too, otherwise # it will coredump. lt_prog_compiler_pic_CXX='-fPIC -shared' ;; *) lt_prog_compiler_pic_CXX='-fPIC' ;; esac else case $host_os in aix[4-9]*) # All AIX code is PIC. if test ia64 = "$host_cpu"; then # AIX 5 now supports IA64 processor lt_prog_compiler_static_CXX='-Bstatic' else lt_prog_compiler_static_CXX='-bnso -bI:/lib/syscalls.exp' fi ;; chorus*) case $cc_basename in cxch68*) # Green Hills C++ Compiler # _LT_TAGVAR(lt_prog_compiler_static, CXX)="--no_auto_instantiation -u __main -u __premain -u _abort -r $COOL_DIR/lib/libOrb.a $MVME_DIR/lib/CC/libC.a $MVME_DIR/lib/classix/libcx.s.a" ;; esac ;; mingw* | cygwin* | os2* | pw32* | cegcc*) # This hack is so that the source file can tell whether it is being # built for inclusion in a dll (and should export symbols for example). lt_prog_compiler_pic_CXX='-DDLL_EXPORT' ;; dgux*) case $cc_basename in ec++*) lt_prog_compiler_pic_CXX='-KPIC' ;; ghcx*) # Green Hills C++ Compiler lt_prog_compiler_pic_CXX='-pic' ;; *) ;; esac ;; freebsd* | dragonfly*) # FreeBSD uses GNU C++ ;; hpux9* | hpux10* | hpux11*) case $cc_basename in CC*) lt_prog_compiler_wl_CXX='-Wl,' lt_prog_compiler_static_CXX='$wl-a ${wl}archive' if test ia64 != "$host_cpu"; then lt_prog_compiler_pic_CXX='+Z' fi ;; aCC*) lt_prog_compiler_wl_CXX='-Wl,' lt_prog_compiler_static_CXX='$wl-a ${wl}archive' case $host_cpu in hppa*64*|ia64*) # +Z the default ;; *) lt_prog_compiler_pic_CXX='+Z' ;; esac ;; *) ;; esac ;; interix*) # This is c89, which is MS Visual C++ (no shared libs) # Anyone wants to do a port? ;; irix5* | irix6* | nonstopux*) case $cc_basename in CC*) lt_prog_compiler_wl_CXX='-Wl,' lt_prog_compiler_static_CXX='-non_shared' # CC pic flag -KPIC is the default. ;; *) ;; esac ;; linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*) case $cc_basename in KCC*) # KAI C++ Compiler lt_prog_compiler_wl_CXX='--backend -Wl,' lt_prog_compiler_pic_CXX='-fPIC' ;; ecpc* ) # old Intel C++ for x86_64, which still supported -KPIC. lt_prog_compiler_wl_CXX='-Wl,' lt_prog_compiler_pic_CXX='-KPIC' lt_prog_compiler_static_CXX='-static' ;; icpc* ) # Intel C++, used to be incompatible with GCC. # ICC 10 doesn't accept -KPIC any more. lt_prog_compiler_wl_CXX='-Wl,' lt_prog_compiler_pic_CXX='-fPIC' lt_prog_compiler_static_CXX='-static' ;; pgCC* | pgcpp*) # Portland Group C++ compiler lt_prog_compiler_wl_CXX='-Wl,' lt_prog_compiler_pic_CXX='-fpic' lt_prog_compiler_static_CXX='-Bstatic' ;; cxx*) # Compaq C++ # Make sure the PIC flag is empty. It appears that all Alpha # Linux and Compaq Tru64 Unix objects are PIC. lt_prog_compiler_pic_CXX= lt_prog_compiler_static_CXX='-non_shared' ;; xlc* | xlC* | bgxl[cC]* | mpixl[cC]*) # IBM XL 8.0, 9.0 on PPC and BlueGene lt_prog_compiler_wl_CXX='-Wl,' lt_prog_compiler_pic_CXX='-qpic' lt_prog_compiler_static_CXX='-qstaticlink' ;; *) case `$CC -V 2>&1 | sed 5q` in *Sun\ C*) # Sun C++ 5.9 lt_prog_compiler_pic_CXX='-KPIC' lt_prog_compiler_static_CXX='-Bstatic' lt_prog_compiler_wl_CXX='-Qoption ld ' ;; esac ;; esac ;; lynxos*) ;; m88k*) ;; mvs*) case $cc_basename in cxx*) lt_prog_compiler_pic_CXX='-W c,exportall' ;; *) ;; esac ;; netbsd* | netbsdelf*-gnu) ;; *qnx* | *nto*) # QNX uses GNU C++, but need to define -shared option too, otherwise # it will coredump. lt_prog_compiler_pic_CXX='-fPIC -shared' ;; osf3* | osf4* | osf5*) case $cc_basename in KCC*) lt_prog_compiler_wl_CXX='--backend -Wl,' ;; RCC*) # Rational C++ 2.4.1 lt_prog_compiler_pic_CXX='-pic' ;; cxx*) # Digital/Compaq C++ lt_prog_compiler_wl_CXX='-Wl,' # Make sure the PIC flag is empty. It appears that all Alpha # Linux and Compaq Tru64 Unix objects are PIC. lt_prog_compiler_pic_CXX= lt_prog_compiler_static_CXX='-non_shared' ;; *) ;; esac ;; psos*) ;; solaris*) case $cc_basename in CC* | sunCC*) # Sun C++ 4.2, 5.x and Centerline C++ lt_prog_compiler_pic_CXX='-KPIC' lt_prog_compiler_static_CXX='-Bstatic' lt_prog_compiler_wl_CXX='-Qoption ld ' ;; gcx*) # Green Hills C++ Compiler lt_prog_compiler_pic_CXX='-PIC' ;; *) ;; esac ;; sunos4*) case $cc_basename in CC*) # Sun C++ 4.x lt_prog_compiler_pic_CXX='-pic' lt_prog_compiler_static_CXX='-Bstatic' ;; lcc*) # Lucid lt_prog_compiler_pic_CXX='-pic' ;; *) ;; esac ;; sysv5* | unixware* | sco3.2v5* | sco5v6* | OpenUNIX*) case $cc_basename in CC*) lt_prog_compiler_wl_CXX='-Wl,' lt_prog_compiler_pic_CXX='-KPIC' lt_prog_compiler_static_CXX='-Bstatic' ;; esac ;; tandem*) case $cc_basename in NCC*) # NonStop-UX NCC 3.20 lt_prog_compiler_pic_CXX='-KPIC' ;; *) ;; esac ;; vxworks*) ;; *) lt_prog_compiler_can_build_shared_CXX=no ;; esac fi case $host_os in # For platforms that do not support PIC, -DPIC is meaningless: *djgpp*) lt_prog_compiler_pic_CXX= ;; *) lt_prog_compiler_pic_CXX="$lt_prog_compiler_pic_CXX -DPIC" ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $compiler option to produce PIC" >&5 $as_echo_n "checking for $compiler option to produce PIC... " >&6; } if ${lt_cv_prog_compiler_pic_CXX+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_prog_compiler_pic_CXX=$lt_prog_compiler_pic_CXX fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_pic_CXX" >&5 $as_echo "$lt_cv_prog_compiler_pic_CXX" >&6; } lt_prog_compiler_pic_CXX=$lt_cv_prog_compiler_pic_CXX # # Check to make sure the PIC flag actually works. # if test -n "$lt_prog_compiler_pic_CXX"; then { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $compiler PIC flag $lt_prog_compiler_pic_CXX works" >&5 $as_echo_n "checking if $compiler PIC flag $lt_prog_compiler_pic_CXX works... " >&6; } if ${lt_cv_prog_compiler_pic_works_CXX+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_prog_compiler_pic_works_CXX=no ac_outfile=conftest.$ac_objext echo "$lt_simple_compile_test_code" > conftest.$ac_ext lt_compiler_flag="$lt_prog_compiler_pic_CXX -DPIC" ## exclude from sc_useless_quotes_in_assignment # Insert the option either (1) after the last *FLAGS variable, or # (2) before a word containing "conftest.", or (3) at the end. # Note that $ac_compile itself does not contain backslashes and begins # with a dollar sign (not a hyphen), so the echo should work correctly. # The option is referenced via a variable to avoid confusing sed. lt_compile=`echo "$ac_compile" | $SED \ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` (eval echo "\"\$as_me:$LINENO: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings other than the usual output. $ECHO "$_lt_compiler_boilerplate" | $SED '/^$/d' >conftest.exp $SED '/^$/d; /^ *+/d' conftest.err >conftest.er2 if test ! -s conftest.er2 || diff conftest.exp conftest.er2 >/dev/null; then lt_cv_prog_compiler_pic_works_CXX=yes fi fi $RM conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_pic_works_CXX" >&5 $as_echo "$lt_cv_prog_compiler_pic_works_CXX" >&6; } if test yes = "$lt_cv_prog_compiler_pic_works_CXX"; then case $lt_prog_compiler_pic_CXX in "" | " "*) ;; *) lt_prog_compiler_pic_CXX=" $lt_prog_compiler_pic_CXX" ;; esac else lt_prog_compiler_pic_CXX= lt_prog_compiler_can_build_shared_CXX=no fi fi # # Check to make sure the static flag actually works. # wl=$lt_prog_compiler_wl_CXX eval lt_tmp_static_flag=\"$lt_prog_compiler_static_CXX\" { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $compiler static flag $lt_tmp_static_flag works" >&5 $as_echo_n "checking if $compiler static flag $lt_tmp_static_flag works... " >&6; } if ${lt_cv_prog_compiler_static_works_CXX+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_prog_compiler_static_works_CXX=no save_LDFLAGS=$LDFLAGS LDFLAGS="$LDFLAGS $lt_tmp_static_flag" echo "$lt_simple_link_test_code" > conftest.$ac_ext if (eval $ac_link 2>conftest.err) && test -s conftest$ac_exeext; then # The linker can only warn and ignore the option if not recognized # So say no if there are warnings if test -s conftest.err; then # Append any errors to the config.log. cat conftest.err 1>&5 $ECHO "$_lt_linker_boilerplate" | $SED '/^$/d' > conftest.exp $SED '/^$/d; /^ *+/d' conftest.err >conftest.er2 if diff conftest.exp conftest.er2 >/dev/null; then lt_cv_prog_compiler_static_works_CXX=yes fi else lt_cv_prog_compiler_static_works_CXX=yes fi fi $RM -r conftest* LDFLAGS=$save_LDFLAGS fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_static_works_CXX" >&5 $as_echo "$lt_cv_prog_compiler_static_works_CXX" >&6; } if test yes = "$lt_cv_prog_compiler_static_works_CXX"; then : else lt_prog_compiler_static_CXX= fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $compiler supports -c -o file.$ac_objext" >&5 $as_echo_n "checking if $compiler supports -c -o file.$ac_objext... " >&6; } if ${lt_cv_prog_compiler_c_o_CXX+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_prog_compiler_c_o_CXX=no $RM -r conftest 2>/dev/null mkdir conftest cd conftest mkdir out echo "$lt_simple_compile_test_code" > conftest.$ac_ext lt_compiler_flag="-o out/conftest2.$ac_objext" # Insert the option either (1) after the last *FLAGS variable, or # (2) before a word containing "conftest.", or (3) at the end. # Note that $ac_compile itself does not contain backslashes and begins # with a dollar sign (not a hyphen), so the echo should work correctly. lt_compile=`echo "$ac_compile" | $SED \ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` (eval echo "\"\$as_me:$LINENO: $lt_compile\"" >&5) (eval "$lt_compile" 2>out/conftest.err) ac_status=$? cat out/conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 if (exit $ac_status) && test -s out/conftest2.$ac_objext then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings $ECHO "$_lt_compiler_boilerplate" | $SED '/^$/d' > out/conftest.exp $SED '/^$/d; /^ *+/d' out/conftest.err >out/conftest.er2 if test ! -s out/conftest.er2 || diff out/conftest.exp out/conftest.er2 >/dev/null; then lt_cv_prog_compiler_c_o_CXX=yes fi fi chmod u+w . 2>&5 $RM conftest* # SGI C++ compiler will create directory out/ii_files/ for # template instantiation test -d out/ii_files && $RM out/ii_files/* && rmdir out/ii_files $RM out/* && rmdir out cd .. $RM -r conftest $RM conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_c_o_CXX" >&5 $as_echo "$lt_cv_prog_compiler_c_o_CXX" >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $compiler supports -c -o file.$ac_objext" >&5 $as_echo_n "checking if $compiler supports -c -o file.$ac_objext... " >&6; } if ${lt_cv_prog_compiler_c_o_CXX+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_prog_compiler_c_o_CXX=no $RM -r conftest 2>/dev/null mkdir conftest cd conftest mkdir out echo "$lt_simple_compile_test_code" > conftest.$ac_ext lt_compiler_flag="-o out/conftest2.$ac_objext" # Insert the option either (1) after the last *FLAGS variable, or # (2) before a word containing "conftest.", or (3) at the end. # Note that $ac_compile itself does not contain backslashes and begins # with a dollar sign (not a hyphen), so the echo should work correctly. lt_compile=`echo "$ac_compile" | $SED \ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` (eval echo "\"\$as_me:$LINENO: $lt_compile\"" >&5) (eval "$lt_compile" 2>out/conftest.err) ac_status=$? cat out/conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 if (exit $ac_status) && test -s out/conftest2.$ac_objext then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings $ECHO "$_lt_compiler_boilerplate" | $SED '/^$/d' > out/conftest.exp $SED '/^$/d; /^ *+/d' out/conftest.err >out/conftest.er2 if test ! -s out/conftest.er2 || diff out/conftest.exp out/conftest.er2 >/dev/null; then lt_cv_prog_compiler_c_o_CXX=yes fi fi chmod u+w . 2>&5 $RM conftest* # SGI C++ compiler will create directory out/ii_files/ for # template instantiation test -d out/ii_files && $RM out/ii_files/* && rmdir out/ii_files $RM out/* && rmdir out cd .. $RM -r conftest $RM conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_c_o_CXX" >&5 $as_echo "$lt_cv_prog_compiler_c_o_CXX" >&6; } hard_links=nottested if test no = "$lt_cv_prog_compiler_c_o_CXX" && test no != "$need_locks"; then # do not overwrite the value of need_locks provided by the user { $as_echo "$as_me:${as_lineno-$LINENO}: checking if we can lock with hard links" >&5 $as_echo_n "checking if we can lock with hard links... " >&6; } hard_links=yes $RM conftest* ln conftest.a conftest.b 2>/dev/null && hard_links=no touch conftest.a ln conftest.a conftest.b 2>&5 || hard_links=no ln conftest.a conftest.b 2>/dev/null && hard_links=no { $as_echo "$as_me:${as_lineno-$LINENO}: result: $hard_links" >&5 $as_echo "$hard_links" >&6; } if test no = "$hard_links"; then { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: '$CC' does not support '-c -o', so 'make -j' may be unsafe" >&5 $as_echo "$as_me: WARNING: '$CC' does not support '-c -o', so 'make -j' may be unsafe" >&2;} need_locks=warn fi else need_locks=no fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether the $compiler linker ($LD) supports shared libraries" >&5 $as_echo_n "checking whether the $compiler linker ($LD) supports shared libraries... " >&6; } export_symbols_cmds_CXX='$NM $libobjs $convenience | $global_symbol_pipe | $SED '\''s/.* //'\'' | sort | uniq > $export_symbols' exclude_expsyms_CXX='_GLOBAL_OFFSET_TABLE_|_GLOBAL__F[ID]_.*' case $host_os in aix[4-9]*) # If we're using GNU nm, then we don't want the "-C" option. # -C means demangle to GNU nm, but means don't demangle to AIX nm. # Without the "-l" option, or with the "-B" option, AIX nm treats # weak defined symbols like other global defined symbols, whereas # GNU nm marks them as "W". # While the 'weak' keyword is ignored in the Export File, we need # it in the Import File for the 'aix-soname' feature, so we have # to replace the "-B" option with "-P" for AIX nm. if $NM -V 2>&1 | $GREP 'GNU' > /dev/null; then export_symbols_cmds_CXX='$NM -Bpg $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W")) && (substr(\$ 3,1,1) != ".")) { if (\$ 2 == "W") { print \$ 3 " weak" } else { print \$ 3 } } }'\'' | sort -u > $export_symbols' else export_symbols_cmds_CXX='`func_echo_all $NM | $SED -e '\''s/B\([^B]*\)$/P\1/'\''` -PCpgl $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) && (substr(\$ 1,1,1) != ".")) { if ((\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) { print \$ 1 " weak" } else { print \$ 1 } } }'\'' | sort -u > $export_symbols' fi ;; pw32*) export_symbols_cmds_CXX=$ltdll_cmds ;; cygwin* | mingw* | cegcc*) case $cc_basename in cl*) exclude_expsyms_CXX='_NULL_IMPORT_DESCRIPTOR|_IMPORT_DESCRIPTOR_.*' ;; *) export_symbols_cmds_CXX='$NM $libobjs $convenience | $global_symbol_pipe | $SED -e '\''/^[BCDGRS][ ]/s/.*[ ]\([^ ]*\)/\1 DATA/;s/^.*[ ]__nm__\([^ ]*\)[ ][^ ]*/\1 DATA/;/^I[ ]/d;/^[AITW][ ]/s/.* //'\'' | sort | uniq > $export_symbols' exclude_expsyms_CXX='[_]+GLOBAL_OFFSET_TABLE_|[_]+GLOBAL__[FID]_.*|[_]+head_[A-Za-z0-9_]+_dll|[A-Za-z0-9_]+_dll_iname' ;; esac ;; linux* | k*bsd*-gnu | gnu*) link_all_deplibs_CXX=no ;; *) export_symbols_cmds_CXX='$NM $libobjs $convenience | $global_symbol_pipe | $SED '\''s/.* //'\'' | sort | uniq > $export_symbols' ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ld_shlibs_CXX" >&5 $as_echo "$ld_shlibs_CXX" >&6; } test no = "$ld_shlibs_CXX" && can_build_shared=no with_gnu_ld_CXX=$with_gnu_ld # # Do we need to explicitly link libc? # case "x$archive_cmds_need_lc_CXX" in x|xyes) # Assume -lc should be added archive_cmds_need_lc_CXX=yes if test yes,yes = "$GCC,$enable_shared"; then case $archive_cmds_CXX in *'~'*) # FIXME: we may have to deal with multi-command sequences. ;; '$CC '*) # Test whether the compiler implicitly links with -lc since on some # systems, -lgcc has to come before -lc. If gcc already passes -lc # to ld, don't add -lc before -lgcc. { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether -lc should be explicitly linked in" >&5 $as_echo_n "checking whether -lc should be explicitly linked in... " >&6; } if ${lt_cv_archive_cmds_need_lc_CXX+:} false; then : $as_echo_n "(cached) " >&6 else $RM conftest* echo "$lt_simple_compile_test_code" > conftest.$ac_ext if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 (eval $ac_compile) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } 2>conftest.err; then soname=conftest lib=conftest libobjs=conftest.$ac_objext deplibs= wl=$lt_prog_compiler_wl_CXX pic_flag=$lt_prog_compiler_pic_CXX compiler_flags=-v linker_flags=-v verstring= output_objdir=. libname=conftest lt_save_allow_undefined_flag=$allow_undefined_flag_CXX allow_undefined_flag_CXX= if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$archive_cmds_CXX 2\>\&1 \| $GREP \" -lc \" \>/dev/null 2\>\&1\""; } >&5 (eval $archive_cmds_CXX 2\>\&1 \| $GREP \" -lc \" \>/dev/null 2\>\&1) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } then lt_cv_archive_cmds_need_lc_CXX=no else lt_cv_archive_cmds_need_lc_CXX=yes fi allow_undefined_flag_CXX=$lt_save_allow_undefined_flag else cat conftest.err 1>&5 fi $RM conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_archive_cmds_need_lc_CXX" >&5 $as_echo "$lt_cv_archive_cmds_need_lc_CXX" >&6; } archive_cmds_need_lc_CXX=$lt_cv_archive_cmds_need_lc_CXX ;; esac fi ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: checking dynamic linker characteristics" >&5 $as_echo_n "checking dynamic linker characteristics... " >&6; } library_names_spec= libname_spec='lib$name' soname_spec= shrext_cmds=.so postinstall_cmds= postuninstall_cmds= finish_cmds= finish_eval= shlibpath_var= shlibpath_overrides_runpath=unknown version_type=none dynamic_linker="$host_os ld.so" sys_lib_dlsearch_path_spec="/lib /usr/lib" need_lib_prefix=unknown hardcode_into_libs=no # when you set need_version to no, make sure it does not cause -set_version # flags to be left without arguments need_version=unknown case $host_os in aix3*) version_type=linux # correct to gnu/linux during the next big refactor library_names_spec='$libname$release$shared_ext$versuffix $libname.a' shlibpath_var=LIBPATH # AIX 3 has no versioning support, so we append a major version to the name. soname_spec='$libname$release$shared_ext$major' ;; aix[4-9]*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no hardcode_into_libs=yes if test ia64 = "$host_cpu"; then # AIX 5 supports IA64 library_names_spec='$libname$release$shared_ext$major $libname$release$shared_ext$versuffix $libname$shared_ext' shlibpath_var=LD_LIBRARY_PATH else # With GCC up to 2.95.x, collect2 would create an import file # for dependence libraries. The import file would start with # the line '#! .'. This would cause the generated library to # depend on '.', always an invalid library. This was fixed in # development snapshots of GCC prior to 3.0. case $host_os in aix4 | aix4.[01] | aix4.[01].*) if { echo '#if __GNUC__ > 2 || (__GNUC__ == 2 && __GNUC_MINOR__ >= 97)' echo ' yes ' echo '#endif'; } | $CC -E - | $GREP yes > /dev/null; then : else can_build_shared=no fi ;; esac # Using Import Files as archive members, it is possible to support # filename-based versioning of shared library archives on AIX. While # this would work for both with and without runtime linking, it will # prevent static linking of such archives. So we do filename-based # shared library versioning with .so extension only, which is used # when both runtime linking and shared linking is enabled. # Unfortunately, runtime linking may impact performance, so we do # not want this to be the default eventually. Also, we use the # versioned .so libs for executables only if there is the -brtl # linker flag in LDFLAGS as well, or --with-aix-soname=svr4 only. # To allow for filename-based versioning support, we need to create # libNAME.so.V as an archive file, containing: # *) an Import File, referring to the versioned filename of the # archive as well as the shared archive member, telling the # bitwidth (32 or 64) of that shared object, and providing the # list of exported symbols of that shared object, eventually # decorated with the 'weak' keyword # *) the shared object with the F_LOADONLY flag set, to really avoid # it being seen by the linker. # At run time we better use the real file rather than another symlink, # but for link time we create the symlink libNAME.so -> libNAME.so.V case $with_aix_soname,$aix_use_runtimelinking in # AIX (on Power*) has no versioning support, so currently we cannot hardcode correct # soname into executable. Probably we can add versioning support to # collect2, so additional links can be useful in future. aix,yes) # traditional libtool dynamic_linker='AIX unversionable lib.so' # If using run time linking (on AIX 4.2 or later) use lib.so # instead of lib.a to let people know that these are not # typical AIX shared libraries. library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' ;; aix,no) # traditional AIX only dynamic_linker='AIX lib.a(lib.so.V)' # We preserve .a as extension for shared libraries through AIX4.2 # and later when we are not doing run time linking. library_names_spec='$libname$release.a $libname.a' soname_spec='$libname$release$shared_ext$major' ;; svr4,*) # full svr4 only dynamic_linker="AIX lib.so.V($shared_archive_member_spec.o)" library_names_spec='$libname$release$shared_ext$major $libname$shared_ext' # We do not specify a path in Import Files, so LIBPATH fires. shlibpath_overrides_runpath=yes ;; *,yes) # both, prefer svr4 dynamic_linker="AIX lib.so.V($shared_archive_member_spec.o), lib.a(lib.so.V)" library_names_spec='$libname$release$shared_ext$major $libname$shared_ext' # unpreferred sharedlib libNAME.a needs extra handling postinstall_cmds='test -n "$linkname" || linkname="$realname"~func_stripname "" ".so" "$linkname"~$install_shared_prog "$dir/$func_stripname_result.$libext" "$destdir/$func_stripname_result.$libext"~test -z "$tstripme" || test -z "$striplib" || $striplib "$destdir/$func_stripname_result.$libext"' postuninstall_cmds='for n in $library_names $old_library; do :; done~func_stripname "" ".so" "$n"~test "$func_stripname_result" = "$n" || func_append rmfiles " $odir/$func_stripname_result.$libext"' # We do not specify a path in Import Files, so LIBPATH fires. shlibpath_overrides_runpath=yes ;; *,no) # both, prefer aix dynamic_linker="AIX lib.a(lib.so.V), lib.so.V($shared_archive_member_spec.o)" library_names_spec='$libname$release.a $libname.a' soname_spec='$libname$release$shared_ext$major' # unpreferred sharedlib libNAME.so.V and symlink libNAME.so need extra handling postinstall_cmds='test -z "$dlname" || $install_shared_prog $dir/$dlname $destdir/$dlname~test -z "$tstripme" || test -z "$striplib" || $striplib $destdir/$dlname~test -n "$linkname" || linkname=$realname~func_stripname "" ".a" "$linkname"~(cd "$destdir" && $LN_S -f $dlname $func_stripname_result.so)' postuninstall_cmds='test -z "$dlname" || func_append rmfiles " $odir/$dlname"~for n in $old_library $library_names; do :; done~func_stripname "" ".a" "$n"~func_append rmfiles " $odir/$func_stripname_result.so"' ;; esac shlibpath_var=LIBPATH fi ;; amigaos*) case $host_cpu in powerpc) # Since July 2007 AmigaOS4 officially supports .so libraries. # When compiling the executable, add -use-dynld -Lsobjs: to the compileline. library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' ;; m68k) library_names_spec='$libname.ixlibrary $libname.a' # Create ${libname}_ixlibrary.a entries in /sys/libs. finish_eval='for lib in `ls $libdir/*.ixlibrary 2>/dev/null`; do libname=`func_echo_all "$lib" | $SED '\''s%^.*/\([^/]*\)\.ixlibrary$%\1%'\''`; $RM /sys/libs/${libname}_ixlibrary.a; $show "cd /sys/libs && $LN_S $lib ${libname}_ixlibrary.a"; cd /sys/libs && $LN_S $lib ${libname}_ixlibrary.a || exit 1; done' ;; esac ;; beos*) library_names_spec='$libname$shared_ext' dynamic_linker="$host_os ld.so" shlibpath_var=LIBRARY_PATH ;; bsdi[45]*) version_type=linux # correct to gnu/linux during the next big refactor need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' finish_cmds='PATH="\$PATH:/sbin" ldconfig $libdir' shlibpath_var=LD_LIBRARY_PATH sys_lib_search_path_spec="/shlib /usr/lib /usr/X11/lib /usr/contrib/lib /lib /usr/local/lib" sys_lib_dlsearch_path_spec="/shlib /usr/lib /usr/local/lib" # the default ld.so.conf also contains /usr/contrib/lib and # /usr/X11R6/lib (/usr/X11 is a link to /usr/X11R6), but let us allow # libtool to hard-code these into programs ;; cygwin* | mingw* | pw32* | cegcc*) version_type=windows shrext_cmds=.dll need_version=no need_lib_prefix=no case $GCC,$cc_basename in yes,*) # gcc library_names_spec='$libname.dll.a' # DLL is installed to $(libdir)/../bin by postinstall_cmds postinstall_cmds='base_file=`basename \$file`~ dlpath=`$SHELL 2>&1 -c '\''. $dir/'\''\$base_file'\''i; echo \$dlname'\''`~ dldir=$destdir/`dirname \$dlpath`~ test -d \$dldir || mkdir -p \$dldir~ $install_prog $dir/$dlname \$dldir/$dlname~ chmod a+x \$dldir/$dlname~ if test -n '\''$stripme'\'' && test -n '\''$striplib'\''; then eval '\''$striplib \$dldir/$dlname'\'' || exit \$?; fi' postuninstall_cmds='dldll=`$SHELL 2>&1 -c '\''. $file; echo \$dlname'\''`~ dlpath=$dir/\$dldll~ $RM \$dlpath' shlibpath_overrides_runpath=yes case $host_os in cygwin*) # Cygwin DLLs use 'cyg' prefix rather than 'lib' soname_spec='`echo $libname | sed -e 's/^lib/cyg/'``echo $release | $SED -e 's/[.]/-/g'`$versuffix$shared_ext' ;; mingw* | cegcc*) # MinGW DLLs use traditional 'lib' prefix soname_spec='$libname`echo $release | $SED -e 's/[.]/-/g'`$versuffix$shared_ext' ;; pw32*) # pw32 DLLs use 'pw' prefix rather than 'lib' library_names_spec='`echo $libname | sed -e 's/^lib/pw/'``echo $release | $SED -e 's/[.]/-/g'`$versuffix$shared_ext' ;; esac dynamic_linker='Win32 ld.exe' ;; *,cl*) # Native MSVC libname_spec='$name' soname_spec='$libname`echo $release | $SED -e 's/[.]/-/g'`$versuffix$shared_ext' library_names_spec='$libname.dll.lib' case $build_os in mingw*) sys_lib_search_path_spec= lt_save_ifs=$IFS IFS=';' for lt_path in $LIB do IFS=$lt_save_ifs # Let DOS variable expansion print the short 8.3 style file name. lt_path=`cd "$lt_path" 2>/dev/null && cmd //C "for %i in (".") do @echo %~si"` sys_lib_search_path_spec="$sys_lib_search_path_spec $lt_path" done IFS=$lt_save_ifs # Convert to MSYS style. sys_lib_search_path_spec=`$ECHO "$sys_lib_search_path_spec" | sed -e 's|\\\\|/|g' -e 's| \\([a-zA-Z]\\):| /\\1|g' -e 's|^ ||'` ;; cygwin*) # Convert to unix form, then to dos form, then back to unix form # but this time dos style (no spaces!) so that the unix form looks # like /cygdrive/c/PROGRA~1:/cygdr... sys_lib_search_path_spec=`cygpath --path --unix "$LIB"` sys_lib_search_path_spec=`cygpath --path --dos "$sys_lib_search_path_spec" 2>/dev/null` sys_lib_search_path_spec=`cygpath --path --unix "$sys_lib_search_path_spec" | $SED -e "s/$PATH_SEPARATOR/ /g"` ;; *) sys_lib_search_path_spec=$LIB if $ECHO "$sys_lib_search_path_spec" | $GREP ';[c-zC-Z]:/' >/dev/null; then # It is most probably a Windows format PATH. sys_lib_search_path_spec=`$ECHO "$sys_lib_search_path_spec" | $SED -e 's/;/ /g'` else sys_lib_search_path_spec=`$ECHO "$sys_lib_search_path_spec" | $SED -e "s/$PATH_SEPARATOR/ /g"` fi # FIXME: find the short name or the path components, as spaces are # common. (e.g. "Program Files" -> "PROGRA~1") ;; esac # DLL is installed to $(libdir)/../bin by postinstall_cmds postinstall_cmds='base_file=`basename \$file`~ dlpath=`$SHELL 2>&1 -c '\''. $dir/'\''\$base_file'\''i; echo \$dlname'\''`~ dldir=$destdir/`dirname \$dlpath`~ test -d \$dldir || mkdir -p \$dldir~ $install_prog $dir/$dlname \$dldir/$dlname' postuninstall_cmds='dldll=`$SHELL 2>&1 -c '\''. $file; echo \$dlname'\''`~ dlpath=$dir/\$dldll~ $RM \$dlpath' shlibpath_overrides_runpath=yes dynamic_linker='Win32 link.exe' ;; *) # Assume MSVC wrapper library_names_spec='$libname`echo $release | $SED -e 's/[.]/-/g'`$versuffix$shared_ext $libname.lib' dynamic_linker='Win32 ld.exe' ;; esac # FIXME: first we should search . and the directory the executable is in shlibpath_var=PATH ;; darwin* | rhapsody*) dynamic_linker="$host_os dyld" version_type=darwin need_lib_prefix=no need_version=no library_names_spec='$libname$release$major$shared_ext $libname$shared_ext' soname_spec='$libname$release$major$shared_ext' shlibpath_overrides_runpath=yes shlibpath_var=DYLD_LIBRARY_PATH shrext_cmds='`test .$module = .yes && echo .so || echo .dylib`' sys_lib_dlsearch_path_spec='/usr/local/lib /lib /usr/lib' ;; dgux*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' shlibpath_var=LD_LIBRARY_PATH ;; freebsd* | dragonfly*) # DragonFly does not have aout. When/if they implement a new # versioning mechanism, adjust this. if test -x /usr/bin/objformat; then objformat=`/usr/bin/objformat` else case $host_os in freebsd[23].*) objformat=aout ;; *) objformat=elf ;; esac fi version_type=freebsd-$objformat case $version_type in freebsd-elf*) library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' need_version=no need_lib_prefix=no ;; freebsd-*) library_names_spec='$libname$release$shared_ext$versuffix $libname$shared_ext$versuffix' need_version=yes ;; esac shlibpath_var=LD_LIBRARY_PATH case $host_os in freebsd2.*) shlibpath_overrides_runpath=yes ;; freebsd3.[01]* | freebsdelf3.[01]*) shlibpath_overrides_runpath=yes hardcode_into_libs=yes ;; freebsd3.[2-9]* | freebsdelf3.[2-9]* | \ freebsd4.[0-5] | freebsdelf4.[0-5] | freebsd4.1.1 | freebsdelf4.1.1) shlibpath_overrides_runpath=no hardcode_into_libs=yes ;; *) # from 4.6 on, and DragonFly shlibpath_overrides_runpath=yes hardcode_into_libs=yes ;; esac ;; haiku*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no dynamic_linker="$host_os runtime_loader" library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' shlibpath_var=LIBRARY_PATH shlibpath_overrides_runpath=no sys_lib_dlsearch_path_spec='/boot/home/config/lib /boot/common/lib /boot/system/lib' hardcode_into_libs=yes ;; hpux9* | hpux10* | hpux11*) # Give a soname corresponding to the major version so that dld.sl refuses to # link against other versions. version_type=sunos need_lib_prefix=no need_version=no case $host_cpu in ia64*) shrext_cmds='.so' hardcode_into_libs=yes dynamic_linker="$host_os dld.so" shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes # Unless +noenvvar is specified. library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' if test 32 = "$HPUX_IA64_MODE"; then sys_lib_search_path_spec="/usr/lib/hpux32 /usr/local/lib/hpux32 /usr/local/lib" sys_lib_dlsearch_path_spec=/usr/lib/hpux32 else sys_lib_search_path_spec="/usr/lib/hpux64 /usr/local/lib/hpux64" sys_lib_dlsearch_path_spec=/usr/lib/hpux64 fi ;; hppa*64*) shrext_cmds='.sl' hardcode_into_libs=yes dynamic_linker="$host_os dld.sl" shlibpath_var=LD_LIBRARY_PATH # How should we handle SHLIB_PATH shlibpath_overrides_runpath=yes # Unless +noenvvar is specified. library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' sys_lib_search_path_spec="/usr/lib/pa20_64 /usr/ccs/lib/pa20_64" sys_lib_dlsearch_path_spec=$sys_lib_search_path_spec ;; *) shrext_cmds='.sl' dynamic_linker="$host_os dld.sl" shlibpath_var=SHLIB_PATH shlibpath_overrides_runpath=no # +s is required to enable SHLIB_PATH library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' ;; esac # HP-UX runs *really* slowly unless shared libraries are mode 555, ... postinstall_cmds='chmod 555 $lib' # or fails outright, so override atomically: install_override_mode=555 ;; interix[3-9]*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' dynamic_linker='Interix 3.x ld.so.1 (PE, like ELF)' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=no hardcode_into_libs=yes ;; irix5* | irix6* | nonstopux*) case $host_os in nonstopux*) version_type=nonstopux ;; *) if test yes = "$lt_cv_prog_gnu_ld"; then version_type=linux # correct to gnu/linux during the next big refactor else version_type=irix fi ;; esac need_lib_prefix=no need_version=no soname_spec='$libname$release$shared_ext$major' library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$release$shared_ext $libname$shared_ext' case $host_os in irix5* | nonstopux*) libsuff= shlibsuff= ;; *) case $LD in # libtool.m4 will add one of these switches to LD *-32|*"-32 "|*-melf32bsmip|*"-melf32bsmip ") libsuff= shlibsuff= libmagic=32-bit;; *-n32|*"-n32 "|*-melf32bmipn32|*"-melf32bmipn32 ") libsuff=32 shlibsuff=N32 libmagic=N32;; *-64|*"-64 "|*-melf64bmip|*"-melf64bmip ") libsuff=64 shlibsuff=64 libmagic=64-bit;; *) libsuff= shlibsuff= libmagic=never-match;; esac ;; esac shlibpath_var=LD_LIBRARY${shlibsuff}_PATH shlibpath_overrides_runpath=no sys_lib_search_path_spec="/usr/lib$libsuff /lib$libsuff /usr/local/lib$libsuff" sys_lib_dlsearch_path_spec="/usr/lib$libsuff /lib$libsuff" hardcode_into_libs=yes ;; # No shared lib support for Linux oldld, aout, or coff. linux*oldld* | linux*aout* | linux*coff*) dynamic_linker=no ;; linux*android*) version_type=none # Android doesn't support versioned libraries. need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext' soname_spec='$libname$release$shared_ext' finish_cmds= shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes # This implies no fast_install, which is unacceptable. # Some rework will be needed to allow for fast_install # before this can be enabled. hardcode_into_libs=yes dynamic_linker='Android linker' # Don't embed -rpath directories since the linker doesn't support them. hardcode_libdir_flag_spec_CXX='-L$libdir' ;; # This must be glibc/ELF. linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' finish_cmds='PATH="\$PATH:/sbin" ldconfig -n $libdir' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=no # Some binutils ld are patched to set DT_RUNPATH if ${lt_cv_shlibpath_overrides_runpath+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_shlibpath_overrides_runpath=no save_LDFLAGS=$LDFLAGS save_libdir=$libdir eval "libdir=/foo; wl=\"$lt_prog_compiler_wl_CXX\"; \ LDFLAGS=\"\$LDFLAGS $hardcode_libdir_flag_spec_CXX\"" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_cxx_try_link "$LINENO"; then : if ($OBJDUMP -p conftest$ac_exeext) 2>/dev/null | grep "RUNPATH.*$libdir" >/dev/null; then : lt_cv_shlibpath_overrides_runpath=yes fi fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LDFLAGS=$save_LDFLAGS libdir=$save_libdir fi shlibpath_overrides_runpath=$lt_cv_shlibpath_overrides_runpath # This implies no fast_install, which is unacceptable. # Some rework will be needed to allow for fast_install # before this can be enabled. hardcode_into_libs=yes # Ideally, we could use ldconfig to report *all* directores which are # searched for libraries, however this is still not possible. Aside from not # being certain /sbin/ldconfig is available, command # 'ldconfig -N -X -v | grep ^/' on 64bit Fedora does not report /usr/lib64, # even though it is searched at run-time. Try to do the best guess by # appending ld.so.conf contents (and includes) to the search path. if test -f /etc/ld.so.conf; then lt_ld_extra=`awk '/^include / { system(sprintf("cd /etc; cat %s 2>/dev/null", \$2)); skip = 1; } { if (!skip) print \$0; skip = 0; }' < /etc/ld.so.conf | $SED -e 's/#.*//;/^[ ]*hwcap[ ]/d;s/[:, ]/ /g;s/=[^=]*$//;s/=[^= ]* / /g;s/"//g;/^$/d' | tr '\n' ' '` sys_lib_dlsearch_path_spec="/lib /usr/lib $lt_ld_extra" fi # We used to test for /lib/ld.so.1 and disable shared libraries on # powerpc, because MkLinux only supported shared libraries with the # GNU dynamic linker. Since this was broken with cross compilers, # most powerpc-linux boxes support dynamic linking these days and # people can always --disable-shared, the test was removed, and we # assume the GNU/Linux dynamic linker is in use. dynamic_linker='GNU/Linux ld.so' ;; netbsdelf*-gnu) version_type=linux need_lib_prefix=no need_version=no library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major ${libname}${shared_ext}' soname_spec='${libname}${release}${shared_ext}$major' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=no hardcode_into_libs=yes dynamic_linker='NetBSD ld.elf_so' ;; netbsd*) version_type=sunos need_lib_prefix=no need_version=no if echo __ELF__ | $CC -E - | $GREP __ELF__ >/dev/null; then library_names_spec='$libname$release$shared_ext$versuffix $libname$shared_ext$versuffix' finish_cmds='PATH="\$PATH:/sbin" ldconfig -m $libdir' dynamic_linker='NetBSD (a.out) ld.so' else library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' dynamic_linker='NetBSD ld.elf_so' fi shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes hardcode_into_libs=yes ;; newsos6) version_type=linux # correct to gnu/linux during the next big refactor library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes ;; *nto* | *qnx*) version_type=qnx need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=no hardcode_into_libs=yes dynamic_linker='ldqnx.so' ;; openbsd* | bitrig*) version_type=sunos sys_lib_dlsearch_path_spec=/usr/lib need_lib_prefix=no if test -z "`echo __ELF__ | $CC -E - | $GREP __ELF__`"; then need_version=no else need_version=yes fi library_names_spec='$libname$release$shared_ext$versuffix $libname$shared_ext$versuffix' finish_cmds='PATH="\$PATH:/sbin" ldconfig -m $libdir' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes ;; os2*) libname_spec='$name' version_type=windows shrext_cmds=.dll need_version=no need_lib_prefix=no # OS/2 can only load a DLL with a base name of 8 characters or less. soname_spec='`test -n "$os2dllname" && libname="$os2dllname"; v=$($ECHO $release$versuffix | tr -d .-); n=$($ECHO $libname | cut -b -$((8 - ${#v})) | tr . _); $ECHO $n$v`$shared_ext' library_names_spec='${libname}_dll.$libext' dynamic_linker='OS/2 ld.exe' shlibpath_var=BEGINLIBPATH sys_lib_search_path_spec="/lib /usr/lib /usr/local/lib" sys_lib_dlsearch_path_spec=$sys_lib_search_path_spec postinstall_cmds='base_file=`basename \$file`~ dlpath=`$SHELL 2>&1 -c '\''. $dir/'\''\$base_file'\''i; $ECHO \$dlname'\''`~ dldir=$destdir/`dirname \$dlpath`~ test -d \$dldir || mkdir -p \$dldir~ $install_prog $dir/$dlname \$dldir/$dlname~ chmod a+x \$dldir/$dlname~ if test -n '\''$stripme'\'' && test -n '\''$striplib'\''; then eval '\''$striplib \$dldir/$dlname'\'' || exit \$?; fi' postuninstall_cmds='dldll=`$SHELL 2>&1 -c '\''. $file; $ECHO \$dlname'\''`~ dlpath=$dir/\$dldll~ $RM \$dlpath' ;; osf3* | osf4* | osf5*) version_type=osf need_lib_prefix=no need_version=no soname_spec='$libname$release$shared_ext$major' library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' shlibpath_var=LD_LIBRARY_PATH sys_lib_search_path_spec="/usr/shlib /usr/ccs/lib /usr/lib/cmplrs/cc /usr/lib /usr/local/lib /var/shlib" sys_lib_dlsearch_path_spec=$sys_lib_search_path_spec ;; rdos*) dynamic_linker=no ;; solaris*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes hardcode_into_libs=yes # ldd complains unless libraries are executable postinstall_cmds='chmod +x $lib' ;; sunos4*) version_type=sunos library_names_spec='$libname$release$shared_ext$versuffix $libname$shared_ext$versuffix' finish_cmds='PATH="\$PATH:/usr/etc" ldconfig $libdir' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes if test yes = "$with_gnu_ld"; then need_lib_prefix=no fi need_version=yes ;; sysv4 | sysv4.3*) version_type=linux # correct to gnu/linux during the next big refactor library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' shlibpath_var=LD_LIBRARY_PATH case $host_vendor in sni) shlibpath_overrides_runpath=no need_lib_prefix=no runpath_var=LD_RUN_PATH ;; siemens) need_lib_prefix=no ;; motorola) need_lib_prefix=no need_version=no shlibpath_overrides_runpath=no sys_lib_search_path_spec='/lib /usr/lib /usr/ccs/lib' ;; esac ;; sysv4*MP*) if test -d /usr/nec; then version_type=linux # correct to gnu/linux during the next big refactor library_names_spec='$libname$shared_ext.$versuffix $libname$shared_ext.$major $libname$shared_ext' soname_spec='$libname$shared_ext.$major' shlibpath_var=LD_LIBRARY_PATH fi ;; sysv5* | sco3.2v5* | sco5v6* | unixware* | OpenUNIX* | sysv4*uw2*) version_type=sco need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes hardcode_into_libs=yes if test yes = "$with_gnu_ld"; then sys_lib_search_path_spec='/usr/local/lib /usr/gnu/lib /usr/ccs/lib /usr/lib /lib' else sys_lib_search_path_spec='/usr/ccs/lib /usr/lib' case $host_os in sco3.2v5*) sys_lib_search_path_spec="$sys_lib_search_path_spec /lib" ;; esac fi sys_lib_dlsearch_path_spec='/usr/lib' ;; tpf*) # TPF is a cross-target only. Preferred cross-host = GNU/Linux. version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=no hardcode_into_libs=yes ;; uts4*) version_type=linux # correct to gnu/linux during the next big refactor library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' soname_spec='$libname$release$shared_ext$major' shlibpath_var=LD_LIBRARY_PATH ;; *) dynamic_linker=no ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: result: $dynamic_linker" >&5 $as_echo "$dynamic_linker" >&6; } test no = "$dynamic_linker" && can_build_shared=no variables_saved_for_relink="PATH $shlibpath_var $runpath_var" if test yes = "$GCC"; then variables_saved_for_relink="$variables_saved_for_relink GCC_EXEC_PREFIX COMPILER_PATH LIBRARY_PATH" fi if test set = "${lt_cv_sys_lib_search_path_spec+set}"; then sys_lib_search_path_spec=$lt_cv_sys_lib_search_path_spec fi if test set = "${lt_cv_sys_lib_dlsearch_path_spec+set}"; then sys_lib_dlsearch_path_spec=$lt_cv_sys_lib_dlsearch_path_spec fi # remember unaugmented sys_lib_dlsearch_path content for libtool script decls... configure_time_dlsearch_path=$sys_lib_dlsearch_path_spec # ... but it needs LT_SYS_LIBRARY_PATH munging for other configure-time code func_munge_path_list sys_lib_dlsearch_path_spec "$LT_SYS_LIBRARY_PATH" # to be used as default LT_SYS_LIBRARY_PATH value in generated libtool configure_time_lt_sys_library_path=$LT_SYS_LIBRARY_PATH { $as_echo "$as_me:${as_lineno-$LINENO}: checking how to hardcode library paths into programs" >&5 $as_echo_n "checking how to hardcode library paths into programs... " >&6; } hardcode_action_CXX= if test -n "$hardcode_libdir_flag_spec_CXX" || test -n "$runpath_var_CXX" || test yes = "$hardcode_automatic_CXX"; then # We can hardcode non-existent directories. if test no != "$hardcode_direct_CXX" && # If the only mechanism to avoid hardcoding is shlibpath_var, we # have to relink, otherwise we might link with an installed library # when we should be linking with a yet-to-be-installed one ## test no != "$_LT_TAGVAR(hardcode_shlibpath_var, CXX)" && test no != "$hardcode_minus_L_CXX"; then # Linking always hardcodes the temporary library directory. hardcode_action_CXX=relink else # We can link without hardcoding, and we can hardcode nonexisting dirs. hardcode_action_CXX=immediate fi else # We cannot hardcode anything, or else we can only hardcode existing # directories. hardcode_action_CXX=unsupported fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $hardcode_action_CXX" >&5 $as_echo "$hardcode_action_CXX" >&6; } if test relink = "$hardcode_action_CXX" || test yes = "$inherit_rpath_CXX"; then # Fast installation is not supported enable_fast_install=no elif test yes = "$shlibpath_overrides_runpath" || test no = "$enable_shared"; then # Fast installation is not necessary enable_fast_install=needless fi fi # test -n "$compiler" CC=$lt_save_CC CFLAGS=$lt_save_CFLAGS LDCXX=$LD LD=$lt_save_LD GCC=$lt_save_GCC with_gnu_ld=$lt_save_with_gnu_ld lt_cv_path_LDCXX=$lt_cv_path_LD lt_cv_path_LD=$lt_save_path_LD lt_cv_prog_gnu_ldcxx=$lt_cv_prog_gnu_ld lt_cv_prog_gnu_ld=$lt_save_with_gnu_ld fi # test yes != "$_lt_caught_CXX_error" ac_ext=cpp ac_cpp='$CXXCPP $CPPFLAGS' ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_cxx_compiler_gnu ac_config_commands="$ac_config_commands libtool" # Only expand once: ax_cxx_compile_cxx11_required=true ac_ext=cpp ac_cpp='$CXXCPP $CPPFLAGS' ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_cxx_compiler_gnu ac_success=no { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether $CXX supports C++11 features by default" >&5 $as_echo_n "checking whether $CXX supports C++11 features by default... " >&6; } if ${ax_cv_cxx_compile_cxx11+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ template struct check { static_assert(sizeof(int) <= sizeof(T), "not big enough"); }; struct Base { virtual void f() {} }; struct Child : public Base { virtual void f() override {} }; typedef check> right_angle_brackets; int a; decltype(a) b; typedef check check_type; check_type c; check_type&& cr = static_cast(c); auto d = a; auto l = [](){}; _ACEOF if ac_fn_cxx_try_compile "$LINENO"; then : ax_cv_cxx_compile_cxx11=yes else ax_cv_cxx_compile_cxx11=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ax_cv_cxx_compile_cxx11" >&5 $as_echo "$ax_cv_cxx_compile_cxx11" >&6; } if test x$ax_cv_cxx_compile_cxx11 = xyes; then ac_success=yes fi if test x$ac_success = xno; then for switch in -std=gnu++11 -std=gnu++0x; do cachevar=`$as_echo "ax_cv_cxx_compile_cxx11_$switch" | $as_tr_sh` { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether $CXX supports C++11 features with $switch" >&5 $as_echo_n "checking whether $CXX supports C++11 features with $switch... " >&6; } if eval \${$cachevar+:} false; then : $as_echo_n "(cached) " >&6 else ac_save_CXXFLAGS="$CXXFLAGS" CXXFLAGS="$CXXFLAGS $switch" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ template struct check { static_assert(sizeof(int) <= sizeof(T), "not big enough"); }; struct Base { virtual void f() {} }; struct Child : public Base { virtual void f() override {} }; typedef check> right_angle_brackets; int a; decltype(a) b; typedef check check_type; check_type c; check_type&& cr = static_cast(c); auto d = a; auto l = [](){}; _ACEOF if ac_fn_cxx_try_compile "$LINENO"; then : eval $cachevar=yes else eval $cachevar=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext CXXFLAGS="$ac_save_CXXFLAGS" fi eval ac_res=\$$cachevar { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 $as_echo "$ac_res" >&6; } if eval test x\$$cachevar = xyes; then CXXFLAGS="$CXXFLAGS $switch" ac_success=yes break fi done fi if test x$ac_success = xno; then for switch in -std=c++11 -std=c++0x; do cachevar=`$as_echo "ax_cv_cxx_compile_cxx11_$switch" | $as_tr_sh` { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether $CXX supports C++11 features with $switch" >&5 $as_echo_n "checking whether $CXX supports C++11 features with $switch... " >&6; } if eval \${$cachevar+:} false; then : $as_echo_n "(cached) " >&6 else ac_save_CXXFLAGS="$CXXFLAGS" CXXFLAGS="$CXXFLAGS $switch" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ template struct check { static_assert(sizeof(int) <= sizeof(T), "not big enough"); }; struct Base { virtual void f() {} }; struct Child : public Base { virtual void f() override {} }; typedef check> right_angle_brackets; int a; decltype(a) b; typedef check check_type; check_type c; check_type&& cr = static_cast(c); auto d = a; auto l = [](){}; _ACEOF if ac_fn_cxx_try_compile "$LINENO"; then : eval $cachevar=yes else eval $cachevar=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext CXXFLAGS="$ac_save_CXXFLAGS" fi eval ac_res=\$$cachevar { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 $as_echo "$ac_res" >&6; } if eval test x\$$cachevar = xyes; then CXXFLAGS="$CXXFLAGS $switch" ac_success=yes break fi done fi ac_ext=cpp ac_cpp='$CXXCPP $CPPFLAGS' ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_cxx_compiler_gnu if test x$ax_cxx_compile_cxx11_required = xtrue; then if test x$ac_success = xno; then as_fn_error $? "*** A compiler with support for C++11 language features is required." "$LINENO" 5 fi else if test x$ac_success = xno; then HAVE_CXX11=0 { $as_echo "$as_me:${as_lineno-$LINENO}: No compiler with C++11 support was found" >&5 $as_echo "$as_me: No compiler with C++11 support was found" >&6;} else HAVE_CXX11=1 $as_echo "#define HAVE_CXX11 1" >>confdefs.h fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for ANSI C header files" >&5 $as_echo_n "checking for ANSI C header files... " >&6; } if ${ac_cv_header_stdc+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include #include #include #include int main () { ; return 0; } _ACEOF if ac_fn_cxx_try_compile "$LINENO"; then : ac_cv_header_stdc=yes else ac_cv_header_stdc=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext if test $ac_cv_header_stdc = yes; then # SunOS 4.x string.h does not declare mem*, contrary to ANSI. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include _ACEOF if (eval "$ac_cpp conftest.$ac_ext") 2>&5 | $EGREP "memchr" >/dev/null 2>&1; then : else ac_cv_header_stdc=no fi rm -f conftest* fi if test $ac_cv_header_stdc = yes; then # ISC 2.0.2 stdlib.h does not declare free, contrary to ANSI. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include _ACEOF if (eval "$ac_cpp conftest.$ac_ext") 2>&5 | $EGREP "free" >/dev/null 2>&1; then : else ac_cv_header_stdc=no fi rm -f conftest* fi if test $ac_cv_header_stdc = yes; then # /bin/cc in Irix-4.0.5 gets non-ANSI ctype macros unless using -ansi. if test "$cross_compiling" = yes; then : : else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include #include #if ((' ' & 0x0FF) == 0x020) # define ISLOWER(c) ('a' <= (c) && (c) <= 'z') # define TOUPPER(c) (ISLOWER(c) ? 'A' + ((c) - 'a') : (c)) #else # define ISLOWER(c) \ (('a' <= (c) && (c) <= 'i') \ || ('j' <= (c) && (c) <= 'r') \ || ('s' <= (c) && (c) <= 'z')) # define TOUPPER(c) (ISLOWER(c) ? ((c) | 0x40) : (c)) #endif #define XOR(e, f) (((e) && !(f)) || (!(e) && (f))) int main () { int i; for (i = 0; i < 256; i++) if (XOR (islower (i), ISLOWER (i)) || toupper (i) != TOUPPER (i)) return 2; return 0; } _ACEOF if ac_fn_cxx_try_run "$LINENO"; then : else ac_cv_header_stdc=no fi rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \ conftest.$ac_objext conftest.beam conftest.$ac_ext fi fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_header_stdc" >&5 $as_echo "$ac_cv_header_stdc" >&6; } if test $ac_cv_header_stdc = yes; then $as_echo "#define STDC_HEADERS 1" >>confdefs.h fi # Check whether --with-boost was given. if test "${with_boost+set}" = set; then : withval=$with_boost; if test "$withval" = "no"; then want_boost="no" elif test "$withval" = "yes"; then want_boost="yes" ac_boost_path="" else want_boost="yes" ac_boost_path="$withval" fi else want_boost="yes" fi # Check whether --with-boost-libdir was given. if test "${with_boost_libdir+set}" = set; then : withval=$with_boost_libdir; if test -d "$withval" then ac_boost_lib_path="$withval" else as_fn_error $? "--with-boost-libdir expected directory name" "$LINENO" 5 fi else ac_boost_lib_path="" fi if test "x$want_boost" = "xyes"; then boost_lib_version_req=1.0 boost_lib_version_req_shorten=`expr $boost_lib_version_req : '\([0-9]*\.[0-9]*\)'` boost_lib_version_req_major=`expr $boost_lib_version_req : '\([0-9]*\)'` boost_lib_version_req_minor=`expr $boost_lib_version_req : '[0-9]*\.\([0-9]*\)'` boost_lib_version_req_sub_minor=`expr $boost_lib_version_req : '[0-9]*\.[0-9]*\.\([0-9]*\)'` if test "x$boost_lib_version_req_sub_minor" = "x" ; then boost_lib_version_req_sub_minor="0" fi WANT_BOOST_VERSION=`expr $boost_lib_version_req_major \* 100000 \+ $boost_lib_version_req_minor \* 100 \+ $boost_lib_version_req_sub_minor` { $as_echo "$as_me:${as_lineno-$LINENO}: checking for boostlib >= $boost_lib_version_req" >&5 $as_echo_n "checking for boostlib >= $boost_lib_version_req... " >&6; } succeeded=no libsubdirs="lib" ax_arch=`uname -m` if test $ax_arch = x86_64 -o $ax_arch = ppc64 -o $ax_arch = s390x -o $ax_arch = sparc64; then libsubdirs="lib64 lib lib64" fi if test "$ac_boost_path" != ""; then BOOST_CPPFLAGS="-I$ac_boost_path/include" for ac_boost_path_tmp in $libsubdirs; do if test -d "$ac_boost_path"/"$ac_boost_path_tmp" ; then BOOST_LDFLAGS="-L$ac_boost_path/$ac_boost_path_tmp" break fi done elif test "$cross_compiling" != yes; then for ac_boost_path_tmp in /usr /usr/local /opt /opt/local ; do if test -d "$ac_boost_path_tmp/include/boost" && test -r "$ac_boost_path_tmp/include/boost"; then for libsubdir in $libsubdirs ; do if ls "$ac_boost_path_tmp/$libsubdir/libboost_"* >/dev/null 2>&1 ; then break; fi done BOOST_LDFLAGS="-L$ac_boost_path_tmp/$libsubdir" BOOST_CPPFLAGS="-I$ac_boost_path_tmp/include" break; fi done fi if test "$ac_boost_lib_path" != ""; then BOOST_LDFLAGS="-L$ac_boost_lib_path" fi CPPFLAGS_SAVED="$CPPFLAGS" CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS" export CPPFLAGS LDFLAGS_SAVED="$LDFLAGS" LDFLAGS="$LDFLAGS $BOOST_LDFLAGS" export LDFLAGS ac_ext=cpp ac_cpp='$CXXCPP $CPPFLAGS' ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_cxx_compiler_gnu cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include int main () { #if BOOST_VERSION >= $WANT_BOOST_VERSION // Everything is okay #else # error Boost version is too old #endif ; return 0; } _ACEOF if ac_fn_cxx_try_compile "$LINENO"; then : { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } succeeded=yes found_system=yes fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext ac_ext=cpp ac_cpp='$CXXCPP $CPPFLAGS' ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_cxx_compiler_gnu if test "x$succeeded" != "xyes"; then _version=0 if test "$ac_boost_path" != ""; then if test -d "$ac_boost_path" && test -r "$ac_boost_path"; then for i in `ls -d $ac_boost_path/include/boost-* 2>/dev/null`; do _version_tmp=`echo $i | sed "s#$ac_boost_path##" | sed 's/\/include\/boost-//' | sed 's/_/./'` V_CHECK=`expr $_version_tmp \> $_version` if test "$V_CHECK" = "1" ; then _version=$_version_tmp fi VERSION_UNDERSCORE=`echo $_version | sed 's/\./_/'` BOOST_CPPFLAGS="-I$ac_boost_path/include/boost-$VERSION_UNDERSCORE" done fi else if test "$cross_compiling" != yes; then for ac_boost_path in /usr /usr/local /opt /opt/local ; do if test -d "$ac_boost_path" && test -r "$ac_boost_path"; then for i in `ls -d $ac_boost_path/include/boost-* 2>/dev/null`; do _version_tmp=`echo $i | sed "s#$ac_boost_path##" | sed 's/\/include\/boost-//' | sed 's/_/./'` V_CHECK=`expr $_version_tmp \> $_version` if test "$V_CHECK" = "1" ; then _version=$_version_tmp best_path=$ac_boost_path fi done fi done VERSION_UNDERSCORE=`echo $_version | sed 's/\./_/'` BOOST_CPPFLAGS="-I$best_path/include/boost-$VERSION_UNDERSCORE" if test "$ac_boost_lib_path" = ""; then for libsubdir in $libsubdirs ; do if ls "$best_path/$libsubdir/libboost_"* >/dev/null 2>&1 ; then break; fi done BOOST_LDFLAGS="-L$best_path/$libsubdir" fi fi if test "x$BOOST_ROOT" != "x"; then for libsubdir in $libsubdirs ; do if ls "$BOOST_ROOT/stage/$libsubdir/libboost_"* >/dev/null 2>&1 ; then break; fi done if test -d "$BOOST_ROOT" && test -r "$BOOST_ROOT" && test -d "$BOOST_ROOT/stage/$libsubdir" && test -r "$BOOST_ROOT/stage/$libsubdir"; then version_dir=`expr //$BOOST_ROOT : '.*/\(.*\)'` stage_version=`echo $version_dir | sed 's/boost_//' | sed 's/_/./g'` stage_version_shorten=`expr $stage_version : '\([0-9]*\.[0-9]*\)'` V_CHECK=`expr $stage_version_shorten \>\= $_version` if test "$V_CHECK" = "1" -a "$ac_boost_lib_path" = "" ; then { $as_echo "$as_me:${as_lineno-$LINENO}: We will use a staged boost library from $BOOST_ROOT" >&5 $as_echo "$as_me: We will use a staged boost library from $BOOST_ROOT" >&6;} BOOST_CPPFLAGS="-I$BOOST_ROOT" BOOST_LDFLAGS="-L$BOOST_ROOT/stage/$libsubdir" fi fi fi fi CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS" export CPPFLAGS LDFLAGS="$LDFLAGS $BOOST_LDFLAGS" export LDFLAGS ac_ext=cpp ac_cpp='$CXXCPP $CPPFLAGS' ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_cxx_compiler_gnu cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include int main () { #if BOOST_VERSION >= $WANT_BOOST_VERSION // Everything is okay #else # error Boost version is too old #endif ; return 0; } _ACEOF if ac_fn_cxx_try_compile "$LINENO"; then : { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } succeeded=yes found_system=yes fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext ac_ext=cpp ac_cpp='$CXXCPP $CPPFLAGS' ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_cxx_compiler_gnu fi if test "$succeeded" != "yes" ; then if test "$_version" = "0" ; then { $as_echo "$as_me:${as_lineno-$LINENO}: We could not detect the boost libraries (version $boost_lib_version_req_shorten or higher). If you have a staged boost library (still not installed) please specify \$BOOST_ROOT in your environment and do not give a PATH to --with-boost option. If you are sure you have boost installed, then check your version number looking in . See http://randspringer.de/boost for more documentation." >&5 $as_echo "$as_me: We could not detect the boost libraries (version $boost_lib_version_req_shorten or higher). If you have a staged boost library (still not installed) please specify \$BOOST_ROOT in your environment and do not give a PATH to --with-boost option. If you are sure you have boost installed, then check your version number looking in . See http://randspringer.de/boost for more documentation." >&6;} else { $as_echo "$as_me:${as_lineno-$LINENO}: Your boost libraries seems to old (version $_version)." >&5 $as_echo "$as_me: Your boost libraries seems to old (version $_version)." >&6;} fi # execute ACTION-IF-NOT-FOUND (if present): as_fn_error $? "Could not find a valid version of boost." "$LINENO" 5 else $as_echo "#define HAVE_BOOST /**/" >>confdefs.h # execute ACTION-IF-FOUND (if present): : fi CPPFLAGS="$CPPFLAGS_SAVED" LDFLAGS="$LDFLAGS_SAVED" fi # Check whether --with-boost-program-options was given. if test "${with_boost_program_options+set}" = set; then : withval=$with_boost_program_options; if test "$withval" = "no"; then want_boost="no" elif test "$withval" = "yes"; then want_boost="yes" ax_boost_user_program_options_lib="" else want_boost="yes" ax_boost_user_program_options_lib="$withval" fi else want_boost="yes" fi if test "x$want_boost" = "xyes"; then export want_boost CPPFLAGS_SAVED="$CPPFLAGS" CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS" export CPPFLAGS LDFLAGS_SAVED="$LDFLAGS" LDFLAGS="$LDFLAGS $BOOST_LDFLAGS" export LDFLAGS { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether the Boost::Program_Options library is available" >&5 $as_echo_n "checking whether the Boost::Program_Options library is available... " >&6; } if ${ax_cv_boost_program_options+:} false; then : $as_echo_n "(cached) " >&6 else ac_ext=cpp ac_cpp='$CXXCPP $CPPFLAGS' ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_cxx_compiler_gnu cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include int main () { boost::program_options::options_description generic("Generic options"); return 0; ; return 0; } _ACEOF if ac_fn_cxx_try_compile "$LINENO"; then : ax_cv_boost_program_options=yes else ax_cv_boost_program_options=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext ac_ext=cpp ac_cpp='$CXXCPP $CPPFLAGS' ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_cxx_compiler_gnu fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ax_cv_boost_program_options" >&5 $as_echo "$ax_cv_boost_program_options" >&6; } if test "$ax_cv_boost_program_options" = yes; then $as_echo "#define HAVE_BOOST_PROGRAM_OPTIONS /**/" >>confdefs.h BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/[^\/]*//'` if test "x$ax_boost_user_program_options_lib" = "x"; then for libextension in `ls $BOOSTLIBDIR/libboost_program_options*.so* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^lib\(boost_program_options.*\)\.so.*$;\1;'` `ls $BOOSTLIBDIR/libboost_program_options*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^lib\(boost_program_options.*\)\.a*$;\1;'` ; do ax_lib=${libextension} as_ac_Lib=`$as_echo "ac_cv_lib_$ax_lib''_exit" | $as_tr_sh` { $as_echo "$as_me:${as_lineno-$LINENO}: checking for exit in -l$ax_lib" >&5 $as_echo_n "checking for exit in -l$ax_lib... " >&6; } if eval \${$as_ac_Lib+:} false; then : $as_echo_n "(cached) " >&6 else ac_check_lib_save_LIBS=$LIBS LIBS="-l$ax_lib $LIBS" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ /* Override any GCC internal prototype to avoid an error. Use char because int might match the return type of a GCC builtin and then its argument prototype would still apply. */ #ifdef __cplusplus extern "C" #endif char exit (); int main () { return exit (); ; return 0; } _ACEOF if ac_fn_cxx_try_link "$LINENO"; then : eval "$as_ac_Lib=yes" else eval "$as_ac_Lib=no" fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi eval ac_res=\$$as_ac_Lib { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 $as_echo "$ac_res" >&6; } if eval test \"x\$"$as_ac_Lib"\" = x"yes"; then : BOOST_PROGRAM_OPTIONS_LIB="-l$ax_lib"; link_program_options="yes"; break else link_program_options="no" fi done if test "x$link_program_options" != "xyes"; then for libextension in `ls $BOOSTLIBDIR/boost_program_options*.dll* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^\(boost_program_options.*\)\.dll.*$;\1;'` `ls $BOOSTLIBDIR/boost_program_options*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^\(boost_program_options.*\)\.a*$;\1;'` ; do ax_lib=${libextension} as_ac_Lib=`$as_echo "ac_cv_lib_$ax_lib''_exit" | $as_tr_sh` { $as_echo "$as_me:${as_lineno-$LINENO}: checking for exit in -l$ax_lib" >&5 $as_echo_n "checking for exit in -l$ax_lib... " >&6; } if eval \${$as_ac_Lib+:} false; then : $as_echo_n "(cached) " >&6 else ac_check_lib_save_LIBS=$LIBS LIBS="-l$ax_lib $LIBS" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ /* Override any GCC internal prototype to avoid an error. Use char because int might match the return type of a GCC builtin and then its argument prototype would still apply. */ #ifdef __cplusplus extern "C" #endif char exit (); int main () { return exit (); ; return 0; } _ACEOF if ac_fn_cxx_try_link "$LINENO"; then : eval "$as_ac_Lib=yes" else eval "$as_ac_Lib=no" fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi eval ac_res=\$$as_ac_Lib { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 $as_echo "$ac_res" >&6; } if eval test \"x\$"$as_ac_Lib"\" = x"yes"; then : BOOST_PROGRAM_OPTIONS_LIB="-l$ax_lib"; link_program_options="yes"; break else link_program_options="no" fi done fi else for ax_lib in $ax_boost_user_program_options_lib boost_program_options-$ax_boost_user_program_options_lib; do as_ac_Lib=`$as_echo "ac_cv_lib_$ax_lib''_main" | $as_tr_sh` { $as_echo "$as_me:${as_lineno-$LINENO}: checking for main in -l$ax_lib" >&5 $as_echo_n "checking for main in -l$ax_lib... " >&6; } if eval \${$as_ac_Lib+:} false; then : $as_echo_n "(cached) " >&6 else ac_check_lib_save_LIBS=$LIBS LIBS="-l$ax_lib $LIBS" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { return main (); ; return 0; } _ACEOF if ac_fn_cxx_try_link "$LINENO"; then : eval "$as_ac_Lib=yes" else eval "$as_ac_Lib=no" fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi eval ac_res=\$$as_ac_Lib { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 $as_echo "$ac_res" >&6; } if eval test \"x\$"$as_ac_Lib"\" = x"yes"; then : BOOST_PROGRAM_OPTIONS_LIB="-l$ax_lib"; link_program_options="yes"; break else link_program_options="no" fi done fi if test "x$ax_lib" = "x"; then as_fn_error $? "Could not find a version of the library!" "$LINENO" 5 fi if test "x$link_program_options" != "xyes"; then as_fn_error $? "Could not link against $ax_lib !" "$LINENO" 5 fi fi CPPFLAGS="$CPPFLAGS_SAVED" LDFLAGS="$LDFLAGS_SAVED" fi # # Handle user hints # { $as_echo "$as_me:${as_lineno-$LINENO}: checking if zlib is wanted" >&5 $as_echo_n "checking if zlib is wanted... " >&6; } # Check whether --with-zlib was given. if test "${with_zlib+set}" = set; then : withval=$with_zlib; if test "$withval" != no ; then zlib_places="/usr/local /usr /opt/local /sw" { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } if test -d "$withval" then zlib_places="$withval $zlib_places" else { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: Sorry, $withval does not exist, checking usual places" >&5 $as_echo "$as_me: WARNING: Sorry, $withval does not exist, checking usual places" >&2;} fi else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi else { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } fi # # Locate zlib, if wanted # if test -n "${zlib_places}" then # check the user supplied or any other more or less 'standard' place: # Most UNIX systems : /usr/local and /usr # MacPorts / Fink on OSX : /opt/local respectively /sw for ZLIB_HOME in ${zlib_places} ; do if test -f "${ZLIB_HOME}/include/zlib.h"; then break; fi ZLIB_HOME="" done # if zlib.h was nowhere to be found, give a notice and bail out if test ! -n "${ZLIB_HOME}"; then as_fn_error $? "No zlib.h in any include directory of ${zlib_places}: either specify a valid zlib installation with --with-zlib=DIR or disable zlib usage with --without-zlib" "$LINENO" 5 fi ZLIB_OLD_LDFLAGS=$LDFLAGS ZLIB_OLD_CPPFLAGS=$LDFLAGS LDFLAGS="$LDFLAGS -L${ZLIB_HOME}/lib" CPPFLAGS="$CPPFLAGS -I${ZLIB_HOME}/include" ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu { $as_echo "$as_me:${as_lineno-$LINENO}: checking for inflateEnd in -lz" >&5 $as_echo_n "checking for inflateEnd in -lz... " >&6; } if ${ac_cv_lib_z_inflateEnd+:} false; then : $as_echo_n "(cached) " >&6 else ac_check_lib_save_LIBS=$LIBS LIBS="-lz $LIBS" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ /* Override any GCC internal prototype to avoid an error. Use char because int might match the return type of a GCC builtin and then its argument prototype would still apply. */ #ifdef __cplusplus extern "C" #endif char inflateEnd (); int main () { return inflateEnd (); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ac_cv_lib_z_inflateEnd=yes else ac_cv_lib_z_inflateEnd=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_z_inflateEnd" >&5 $as_echo "$ac_cv_lib_z_inflateEnd" >&6; } if test "x$ac_cv_lib_z_inflateEnd" = xyes; then : zlib_cv_libz=yes else zlib_cv_libz=no fi ac_fn_c_check_header_mongrel "$LINENO" "zlib.h" "ac_cv_header_zlib_h" "$ac_includes_default" if test "x$ac_cv_header_zlib_h" = xyes; then : zlib_cv_zlib_h=yes else zlib_cv_zlib_h=no fi ac_ext=cpp ac_cpp='$CXXCPP $CPPFLAGS' ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_cxx_compiler_gnu if test "$zlib_cv_libz" = "yes" -a "$zlib_cv_zlib_h" = "yes" then # # If both library and header were found, use them # { $as_echo "$as_me:${as_lineno-$LINENO}: checking for inflateEnd in -lz" >&5 $as_echo_n "checking for inflateEnd in -lz... " >&6; } if ${ac_cv_lib_z_inflateEnd+:} false; then : $as_echo_n "(cached) " >&6 else ac_check_lib_save_LIBS=$LIBS LIBS="-lz $LIBS" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ /* Override any GCC internal prototype to avoid an error. Use char because int might match the return type of a GCC builtin and then its argument prototype would still apply. */ #ifdef __cplusplus extern "C" #endif char inflateEnd (); int main () { return inflateEnd (); ; return 0; } _ACEOF if ac_fn_cxx_try_link "$LINENO"; then : ac_cv_lib_z_inflateEnd=yes else ac_cv_lib_z_inflateEnd=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_z_inflateEnd" >&5 $as_echo "$ac_cv_lib_z_inflateEnd" >&6; } if test "x$ac_cv_lib_z_inflateEnd" = xyes; then : cat >>confdefs.h <<_ACEOF #define HAVE_LIBZ 1 _ACEOF LIBS="-lz $LIBS" fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking zlib in ${ZLIB_HOME}" >&5 $as_echo_n "checking zlib in ${ZLIB_HOME}... " >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: result: ok" >&5 $as_echo "ok" >&6; } else # # If either header or library was not found, revert and bomb # { $as_echo "$as_me:${as_lineno-$LINENO}: checking zlib in ${ZLIB_HOME}" >&5 $as_echo_n "checking zlib in ${ZLIB_HOME}... " >&6; } LDFLAGS="$ZLIB_OLD_LDFLAGS" CPPFLAGS="$ZLIB_OLD_CPPFLAGS" { $as_echo "$as_me:${as_lineno-$LINENO}: result: failed" >&5 $as_echo "failed" >&6; } as_fn_error $? "either specify a valid zlib installation with --with-zlib=DIR or disable zlib usage with --without-zlib" "$LINENO" 5 fi fi ZLIB_CPPFLAGS="-I${ZLIB_HOME}/include" ZLIB_LDFLAGS="-L${ZLIB_HOME}/lib -lz" PTHREAD_LIBS=-lpthread ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu ax_pthread_ok=no # We used to check for pthread.h first, but this fails if pthread.h # requires special compiler flags (e.g. on True64 or Sequent). # It gets checked for in the link test anyway. # First of all, check if the user has set any of the PTHREAD_LIBS, # etcetera environment variables, and if threads linking works using # them: if test x"$PTHREAD_LIBS$PTHREAD_CFLAGS" != x; then save_CFLAGS="$CFLAGS" CFLAGS="$CFLAGS $PTHREAD_CFLAGS" save_LIBS="$LIBS" LIBS="$PTHREAD_LIBS $LIBS" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for pthread_join in LIBS=$PTHREAD_LIBS with CFLAGS=$PTHREAD_CFLAGS" >&5 $as_echo_n "checking for pthread_join in LIBS=$PTHREAD_LIBS with CFLAGS=$PTHREAD_CFLAGS... " >&6; } cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ /* Override any GCC internal prototype to avoid an error. Use char because int might match the return type of a GCC builtin and then its argument prototype would still apply. */ #ifdef __cplusplus extern "C" #endif char pthread_join (); int main () { return pthread_join (); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ax_pthread_ok=yes fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ax_pthread_ok" >&5 $as_echo "$ax_pthread_ok" >&6; } if test x"$ax_pthread_ok" = xno; then PTHREAD_LIBS="" PTHREAD_CFLAGS="" fi LIBS="$save_LIBS" CFLAGS="$save_CFLAGS" fi # We must check for the threads library under a number of different # names; the ordering is very important because some systems # (e.g. DEC) have both -lpthread and -lpthreads, where one of the # libraries is broken (non-POSIX). # Create a list of thread flags to try. Items starting with a "-" are # C compiler flags, and other items are library names, except for "none" # which indicates that we try without any flags at all, and "pthread-config" # which is a program returning the flags for the Pth emulation library. ax_pthread_flags="pthreads none -Kthread -kthread lthread -pthread -pthreads -mthreads pthread --thread-safe -mt pthread-config" # The ordering *is* (sometimes) important. Some notes on the # individual items follow: # pthreads: AIX (must check this before -lpthread) # none: in case threads are in libc; should be tried before -Kthread and # other compiler flags to prevent continual compiler warnings # -Kthread: Sequent (threads in libc, but -Kthread needed for pthread.h) # -kthread: FreeBSD kernel threads (preferred to -pthread since SMP-able) # lthread: LinuxThreads port on FreeBSD (also preferred to -pthread) # -pthread: Linux/gcc (kernel threads), BSD/gcc (userland threads) # -pthreads: Solaris/gcc # -mthreads: Mingw32/gcc, Lynx/gcc # -mt: Sun Workshop C (may only link SunOS threads [-lthread], but it # doesn't hurt to check since this sometimes defines pthreads too; # also defines -D_REENTRANT) # ... -mt is also the pthreads flag for HP/aCC # pthread: Linux, etcetera # --thread-safe: KAI C++ # pthread-config: use pthread-config program (for GNU Pth library) case "${host_cpu}-${host_os}" in *solaris*) # On Solaris (at least, for some versions), libc contains stubbed # (non-functional) versions of the pthreads routines, so link-based # tests will erroneously succeed. (We need to link with -pthreads/-mt/ # -lpthread.) (The stubs are missing pthread_cleanup_push, or rather # a function called by this macro, so we could check for that, but # who knows whether they'll stub that too in a future libc.) So, # we'll just look for -pthreads and -lpthread first: ax_pthread_flags="-pthreads pthread -mt -pthread $ax_pthread_flags" ;; *-darwin*) ax_pthread_flags="-pthread $ax_pthread_flags" ;; esac if test x"$ax_pthread_ok" = xno; then for flag in $ax_pthread_flags; do case $flag in none) { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether pthreads work without any flags" >&5 $as_echo_n "checking whether pthreads work without any flags... " >&6; } ;; -*) { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether pthreads work with $flag" >&5 $as_echo_n "checking whether pthreads work with $flag... " >&6; } PTHREAD_CFLAGS="$flag" ;; pthread-config) # Extract the first word of "pthread-config", so it can be a program name with args. set dummy pthread-config; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ax_pthread_config+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ax_pthread_config"; then ac_cv_prog_ax_pthread_config="$ax_pthread_config" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ax_pthread_config="yes" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS test -z "$ac_cv_prog_ax_pthread_config" && ac_cv_prog_ax_pthread_config="no" fi fi ax_pthread_config=$ac_cv_prog_ax_pthread_config if test -n "$ax_pthread_config"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ax_pthread_config" >&5 $as_echo "$ax_pthread_config" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test x"$ax_pthread_config" = xno; then continue; fi PTHREAD_CFLAGS="`pthread-config --cflags`" PTHREAD_LIBS="`pthread-config --ldflags` `pthread-config --libs`" ;; *) { $as_echo "$as_me:${as_lineno-$LINENO}: checking for the pthreads library -l$flag" >&5 $as_echo_n "checking for the pthreads library -l$flag... " >&6; } PTHREAD_LIBS="-l$flag" ;; esac save_LIBS="$LIBS" save_CFLAGS="$CFLAGS" LIBS="$PTHREAD_LIBS $LIBS" CFLAGS="$CFLAGS $PTHREAD_CFLAGS" # Check for various functions. We must include pthread.h, # since some functions may be macros. (On the Sequent, we # need a special flag -Kthread to make this header compile.) # We check for pthread_join because it is in -lpthread on IRIX # while pthread_create is in libc. We check for pthread_attr_init # due to DEC craziness with -lpthreads. We check for # pthread_cleanup_push because it is one of the few pthread # functions on Solaris that doesn't have a non-functional libc stub. # We try pthread_create on general principles. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include static void routine(void *a) { a = 0; } static void *start_routine(void *a) { return a; } int main () { pthread_t th; pthread_attr_t attr; pthread_create(&th, 0, start_routine, 0); pthread_join(th, 0); pthread_attr_init(&attr); pthread_cleanup_push(routine, 0); pthread_cleanup_pop(0) /* ; */ ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ax_pthread_ok=yes fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS="$save_LIBS" CFLAGS="$save_CFLAGS" { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ax_pthread_ok" >&5 $as_echo "$ax_pthread_ok" >&6; } if test "x$ax_pthread_ok" = xyes; then break; fi PTHREAD_LIBS="" PTHREAD_CFLAGS="" done fi # Various other checks: if test "x$ax_pthread_ok" = xyes; then save_LIBS="$LIBS" LIBS="$PTHREAD_LIBS $LIBS" save_CFLAGS="$CFLAGS" CFLAGS="$CFLAGS $PTHREAD_CFLAGS" # Detect AIX lossage: JOINABLE attribute is called UNDETACHED. { $as_echo "$as_me:${as_lineno-$LINENO}: checking for joinable pthread attribute" >&5 $as_echo_n "checking for joinable pthread attribute... " >&6; } attr_name=unknown for attr in PTHREAD_CREATE_JOINABLE PTHREAD_CREATE_UNDETACHED; do cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include int main () { int attr = $attr; return attr /* ; */ ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : attr_name=$attr; break fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext done { $as_echo "$as_me:${as_lineno-$LINENO}: result: $attr_name" >&5 $as_echo "$attr_name" >&6; } if test "$attr_name" != PTHREAD_CREATE_JOINABLE; then cat >>confdefs.h <<_ACEOF #define PTHREAD_CREATE_JOINABLE $attr_name _ACEOF fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking if more special flags are required for pthreads" >&5 $as_echo_n "checking if more special flags are required for pthreads... " >&6; } flag=no case "${host_cpu}-${host_os}" in *-aix* | *-freebsd* | *-darwin*) flag="-D_THREAD_SAFE";; *-osf* | *-hpux*) flag="-D_REENTRANT";; *solaris*) if test "$GCC" = "yes"; then flag="-D_REENTRANT" else flag="-mt -D_REENTRANT" fi ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: result: ${flag}" >&5 $as_echo "${flag}" >&6; } if test "x$flag" != xno; then PTHREAD_CFLAGS="$flag $PTHREAD_CFLAGS" fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for PTHREAD_PRIO_INHERIT" >&5 $as_echo_n "checking for PTHREAD_PRIO_INHERIT... " >&6; } if ${ax_cv_PTHREAD_PRIO_INHERIT+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include int main () { int i = PTHREAD_PRIO_INHERIT; ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ax_cv_PTHREAD_PRIO_INHERIT=yes else ax_cv_PTHREAD_PRIO_INHERIT=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ax_cv_PTHREAD_PRIO_INHERIT" >&5 $as_echo "$ax_cv_PTHREAD_PRIO_INHERIT" >&6; } if test "x$ax_cv_PTHREAD_PRIO_INHERIT" = "xyes"; then : $as_echo "#define HAVE_PTHREAD_PRIO_INHERIT 1" >>confdefs.h fi LIBS="$save_LIBS" CFLAGS="$save_CFLAGS" # More AIX lossage: must compile with xlc_r or cc_r if test x"$GCC" != xyes; then for ac_prog in xlc_r cc_r do # Extract the first word of "$ac_prog", so it can be a program name with args. set dummy $ac_prog; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_PTHREAD_CC+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$PTHREAD_CC"; then ac_cv_prog_PTHREAD_CC="$PTHREAD_CC" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_PTHREAD_CC="$ac_prog" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi PTHREAD_CC=$ac_cv_prog_PTHREAD_CC if test -n "$PTHREAD_CC"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $PTHREAD_CC" >&5 $as_echo "$PTHREAD_CC" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi test -n "$PTHREAD_CC" && break done test -n "$PTHREAD_CC" || PTHREAD_CC="${CC}" else PTHREAD_CC=$CC fi else PTHREAD_CC="$CC" fi # Finally, execute ACTION-IF-FOUND/ACTION-IF-NOT-FOUND: if test x"$ax_pthread_ok" = xyes; then $as_echo "#define HAVE_PTHREAD 1" >>confdefs.h : else ax_pthread_ok=no as_fn_error $? "Could not find posix thread library." "$LINENO" 5 fi ac_ext=cpp ac_cpp='$CXXCPP $CPPFLAGS' ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CXX -o conftest$ac_exeext $CXXFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_cxx_compiler_gnu nitpick=false # Check whether --enable-nitpicking was given. if test "${enable_nitpicking+set}" = set; then : enableval=$enable_nitpicking; test "$enableval" = "no" || nitpick=true fi if test x$nitpick = xtrue; then NITPICK_TRUE= NITPICK_FALSE='#' else NITPICK_TRUE='#' NITPICK_FALSE= fi parallelize=false # Check whether --enable-parallelization was given. if test "${enable_parallelization+set}" = set; then : enableval=$enable_parallelization; test "$enableval" = "no" || parallelize=true fi if test x$parallelize = xtrue; then PARALLELIZE_TRUE= PARALLELIZE_FALSE='#' else PARALLELIZE_TRUE='#' PARALLELIZE_FALSE= fi profile=false # Check whether --enable-profiling was given. if test "${enable_profiling+set}" = set; then : enableval=$enable_profiling; test "$enableval" = "no" || profile=true fi if test x$profile = xtrue; then PROFILE_TRUE= PROFILE_FALSE='#' else PROFILE_TRUE='#' PROFILE_FALSE= fi vwbug=false # Check whether --enable-debug was given. if test "${enable_debug+set}" = set; then : enableval=$enable_debug; test "$enableval" = "no" || vwbug=true fi if test x$vwbug = xtrue; then VWBUG_TRUE= VWBUG_FALSE='#' else VWBUG_TRUE='#' VWBUG_FALSE= fi clang_libcxx=false # Check whether --enable-libc++ was given. if test "${enable_libc__+set}" = set; then : enableval=$enable_libc__; test "$enableval" = "no" || clang_libcxx=true fi if test x$clang_libcxx = xtrue; then CLANG_LIBCXX_TRUE= CLANG_LIBCXX_FALSE='#' else CLANG_LIBCXX_TRUE='#' CLANG_LIBCXX_FALSE= fi case $host in *-*-freebsd*) freebsd=true ;; *) freebsd=false ;; esac if test x$freebsd = xtrue; then FREEBSD_TRUE= FREEBSD_FALSE='#' else FREEBSD_TRUE='#' FREEBSD_FALSE= fi ac_config_files="$ac_config_files Makefile vowpalwabbit/Makefile cluster/Makefile library/Makefile libvw.pc libvw_c_wrapper.pc" cat >confcache <<\_ACEOF # This file is a shell script that caches the results of configure # tests run on this system so they can be shared between configure # scripts and configure runs, see configure's option --config-cache. # It is not useful on other systems. If it contains results you don't # want to keep, you may remove or edit it. # # config.status only pays attention to the cache file if you give it # the --recheck option to rerun configure. # # `ac_cv_env_foo' variables (set or unset) will be overridden when # loading this file, other *unset* `ac_cv_foo' will be assigned the # following values. _ACEOF # The following way of writing the cache mishandles newlines in values, # but we know of no workaround that is simple, portable, and efficient. # So, we kill variables containing newlines. # Ultrix sh set writes to stderr and can't be redirected directly, # and sets the high bit in the cache file unless we assign to the vars. ( for ac_var in `(set) 2>&1 | sed -n 's/^\([a-zA-Z_][a-zA-Z0-9_]*\)=.*/\1/p'`; do eval ac_val=\$$ac_var case $ac_val in #( *${as_nl}*) case $ac_var in #( *_cv_*) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: cache variable $ac_var contains a newline" >&5 $as_echo "$as_me: WARNING: cache variable $ac_var contains a newline" >&2;} ;; esac case $ac_var in #( _ | IFS | as_nl) ;; #( BASH_ARGV | BASH_SOURCE) eval $ac_var= ;; #( *) { eval $ac_var=; unset $ac_var;} ;; esac ;; esac done (set) 2>&1 | case $as_nl`(ac_space=' '; set) 2>&1` in #( *${as_nl}ac_space=\ *) # `set' does not quote correctly, so add quotes: double-quote # substitution turns \\\\ into \\, and sed turns \\ into \. sed -n \ "s/'/'\\\\''/g; s/^\\([_$as_cr_alnum]*_cv_[_$as_cr_alnum]*\\)=\\(.*\\)/\\1='\\2'/p" ;; #( *) # `set' quotes correctly as required by POSIX, so do not add quotes. sed -n "/^[_$as_cr_alnum]*_cv_[_$as_cr_alnum]*=/p" ;; esac | sort ) | sed ' /^ac_cv_env_/b end t clear :clear s/^\([^=]*\)=\(.*[{}].*\)$/test "${\1+set}" = set || &/ t end s/^\([^=]*\)=\(.*\)$/\1=${\1=\2}/ :end' >>confcache if diff "$cache_file" confcache >/dev/null 2>&1; then :; else if test -w "$cache_file"; then if test "x$cache_file" != "x/dev/null"; then { $as_echo "$as_me:${as_lineno-$LINENO}: updating cache $cache_file" >&5 $as_echo "$as_me: updating cache $cache_file" >&6;} if test ! -f "$cache_file" || test -h "$cache_file"; then cat confcache >"$cache_file" else case $cache_file in #( */* | ?:*) mv -f confcache "$cache_file"$$ && mv -f "$cache_file"$$ "$cache_file" ;; #( *) mv -f confcache "$cache_file" ;; esac fi fi else { $as_echo "$as_me:${as_lineno-$LINENO}: not updating unwritable cache $cache_file" >&5 $as_echo "$as_me: not updating unwritable cache $cache_file" >&6;} fi fi rm -f confcache test "x$prefix" = xNONE && prefix=$ac_default_prefix # Let make expand exec_prefix. test "x$exec_prefix" = xNONE && exec_prefix='${prefix}' DEFS=-DHAVE_CONFIG_H ac_libobjs= ac_ltlibobjs= U= for ac_i in : $LIBOBJS; do test "x$ac_i" = x: && continue # 1. Remove the extension, and $U if already installed. ac_script='s/\$U\././;s/\.o$//;s/\.obj$//' ac_i=`$as_echo "$ac_i" | sed "$ac_script"` # 2. Prepend LIBOBJDIR. When used with automake>=1.10 LIBOBJDIR # will be set to the directory where LIBOBJS objects are built. as_fn_append ac_libobjs " \${LIBOBJDIR}$ac_i\$U.$ac_objext" as_fn_append ac_ltlibobjs " \${LIBOBJDIR}$ac_i"'$U.lo' done LIBOBJS=$ac_libobjs LTLIBOBJS=$ac_ltlibobjs { $as_echo "$as_me:${as_lineno-$LINENO}: checking that generated files are newer than configure" >&5 $as_echo_n "checking that generated files are newer than configure... " >&6; } if test -n "$am_sleep_pid"; then # Hide warnings about reused PIDs. wait $am_sleep_pid 2>/dev/null fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: done" >&5 $as_echo "done" >&6; } if test -n "$EXEEXT"; then am__EXEEXT_TRUE= am__EXEEXT_FALSE='#' else am__EXEEXT_TRUE='#' am__EXEEXT_FALSE= fi if test -z "${AMDEP_TRUE}" && test -z "${AMDEP_FALSE}"; then as_fn_error $? "conditional \"AMDEP\" was never defined. Usually this means the macro was only invoked conditionally." "$LINENO" 5 fi if test -z "${am__fastdepCXX_TRUE}" && test -z "${am__fastdepCXX_FALSE}"; then as_fn_error $? "conditional \"am__fastdepCXX\" was never defined. Usually this means the macro was only invoked conditionally." "$LINENO" 5 fi if test -z "${am__fastdepCC_TRUE}" && test -z "${am__fastdepCC_FALSE}"; then as_fn_error $? "conditional \"am__fastdepCC\" was never defined. Usually this means the macro was only invoked conditionally." "$LINENO" 5 fi if test -z "${NITPICK_TRUE}" && test -z "${NITPICK_FALSE}"; then as_fn_error $? "conditional \"NITPICK\" was never defined. Usually this means the macro was only invoked conditionally." "$LINENO" 5 fi if test -z "${PARALLELIZE_TRUE}" && test -z "${PARALLELIZE_FALSE}"; then as_fn_error $? "conditional \"PARALLELIZE\" was never defined. Usually this means the macro was only invoked conditionally." "$LINENO" 5 fi if test -z "${PROFILE_TRUE}" && test -z "${PROFILE_FALSE}"; then as_fn_error $? "conditional \"PROFILE\" was never defined. Usually this means the macro was only invoked conditionally." "$LINENO" 5 fi if test -z "${VWBUG_TRUE}" && test -z "${VWBUG_FALSE}"; then as_fn_error $? "conditional \"VWBUG\" was never defined. Usually this means the macro was only invoked conditionally." "$LINENO" 5 fi if test -z "${CLANG_LIBCXX_TRUE}" && test -z "${CLANG_LIBCXX_FALSE}"; then as_fn_error $? "conditional \"CLANG_LIBCXX\" was never defined. Usually this means the macro was only invoked conditionally." "$LINENO" 5 fi if test -z "${FREEBSD_TRUE}" && test -z "${FREEBSD_FALSE}"; then as_fn_error $? "conditional \"FREEBSD\" was never defined. Usually this means the macro was only invoked conditionally." "$LINENO" 5 fi : "${CONFIG_STATUS=./config.status}" ac_write_fail=0 ac_clean_files_save=$ac_clean_files ac_clean_files="$ac_clean_files $CONFIG_STATUS" { $as_echo "$as_me:${as_lineno-$LINENO}: creating $CONFIG_STATUS" >&5 $as_echo "$as_me: creating $CONFIG_STATUS" >&6;} as_write_fail=0 cat >$CONFIG_STATUS <<_ASEOF || as_write_fail=1 #! $SHELL # Generated by $as_me. # Run this file to recreate the current configuration. # Compiler output produced by configure, useful for debugging # configure, is in config.log if it exists. debug=false ac_cs_recheck=false ac_cs_silent=false SHELL=\${CONFIG_SHELL-$SHELL} export SHELL _ASEOF cat >>$CONFIG_STATUS <<\_ASEOF || as_write_fail=1 ## -------------------- ## ## M4sh Initialization. ## ## -------------------- ## # Be more Bourne compatible DUALCASE=1; export DUALCASE # for MKS sh if test -n "${ZSH_VERSION+set}" && (emulate sh) >/dev/null 2>&1; then : emulate sh NULLCMD=: # Pre-4.2 versions of Zsh do word splitting on ${1+"$@"}, which # is contrary to our usage. Disable this feature. alias -g '${1+"$@"}'='"$@"' setopt NO_GLOB_SUBST else case `(set -o) 2>/dev/null` in #( *posix*) : set -o posix ;; #( *) : ;; esac fi as_nl=' ' export as_nl # Printing a long string crashes Solaris 7 /usr/bin/printf. as_echo='\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\' as_echo=$as_echo$as_echo$as_echo$as_echo$as_echo as_echo=$as_echo$as_echo$as_echo$as_echo$as_echo$as_echo # Prefer a ksh shell builtin over an external printf program on Solaris, # but without wasting forks for bash or zsh. if test -z "$BASH_VERSION$ZSH_VERSION" \ && (test "X`print -r -- $as_echo`" = "X$as_echo") 2>/dev/null; then as_echo='print -r --' as_echo_n='print -rn --' elif (test "X`printf %s $as_echo`" = "X$as_echo") 2>/dev/null; then as_echo='printf %s\n' as_echo_n='printf %s' else if test "X`(/usr/ucb/echo -n -n $as_echo) 2>/dev/null`" = "X-n $as_echo"; then as_echo_body='eval /usr/ucb/echo -n "$1$as_nl"' as_echo_n='/usr/ucb/echo -n' else as_echo_body='eval expr "X$1" : "X\\(.*\\)"' as_echo_n_body='eval arg=$1; case $arg in #( *"$as_nl"*) expr "X$arg" : "X\\(.*\\)$as_nl"; arg=`expr "X$arg" : ".*$as_nl\\(.*\\)"`;; esac; expr "X$arg" : "X\\(.*\\)" | tr -d "$as_nl" ' export as_echo_n_body as_echo_n='sh -c $as_echo_n_body as_echo' fi export as_echo_body as_echo='sh -c $as_echo_body as_echo' fi # The user is always right. if test "${PATH_SEPARATOR+set}" != set; then PATH_SEPARATOR=: (PATH='/bin;/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 && { (PATH='/bin:/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 || PATH_SEPARATOR=';' } fi # IFS # We need space, tab and new line, in precisely that order. Quoting is # there to prevent editors from complaining about space-tab. # (If _AS_PATH_WALK were called with IFS unset, it would disable word # splitting by setting IFS to empty value.) IFS=" "" $as_nl" # Find who we are. Look in the path if we contain no directory separator. as_myself= case $0 in #(( *[\\/]* ) as_myself=$0 ;; *) as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. test -r "$as_dir/$0" && as_myself=$as_dir/$0 && break done IFS=$as_save_IFS ;; esac # We did not find ourselves, most probably we were run as `sh COMMAND' # in which case we are not to be found in the path. if test "x$as_myself" = x; then as_myself=$0 fi if test ! -f "$as_myself"; then $as_echo "$as_myself: error: cannot find myself; rerun with an absolute file name" >&2 exit 1 fi # Unset variables that we do not need and which cause bugs (e.g. in # pre-3.0 UWIN ksh). But do not cause bugs in bash 2.01; the "|| exit 1" # suppresses any "Segmentation fault" message there. '((' could # trigger a bug in pdksh 5.2.14. for as_var in BASH_ENV ENV MAIL MAILPATH do eval test x\${$as_var+set} = xset \ && ( (unset $as_var) || exit 1) >/dev/null 2>&1 && unset $as_var || : done PS1='$ ' PS2='> ' PS4='+ ' # NLS nuisances. LC_ALL=C export LC_ALL LANGUAGE=C export LANGUAGE # CDPATH. (unset CDPATH) >/dev/null 2>&1 && unset CDPATH # as_fn_error STATUS ERROR [LINENO LOG_FD] # ---------------------------------------- # Output "`basename $0`: error: ERROR" to stderr. If LINENO and LOG_FD are # provided, also output the error to LOG_FD, referencing LINENO. Then exit the # script with STATUS, using 1 if that was 0. as_fn_error () { as_status=$1; test $as_status -eq 0 && as_status=1 if test "$4"; then as_lineno=${as_lineno-"$3"} as_lineno_stack=as_lineno_stack=$as_lineno_stack $as_echo "$as_me:${as_lineno-$LINENO}: error: $2" >&$4 fi $as_echo "$as_me: error: $2" >&2 as_fn_exit $as_status } # as_fn_error # as_fn_set_status STATUS # ----------------------- # Set $? to STATUS, without forking. as_fn_set_status () { return $1 } # as_fn_set_status # as_fn_exit STATUS # ----------------- # Exit the shell with STATUS, even in a "trap 0" or "set -e" context. as_fn_exit () { set +e as_fn_set_status $1 exit $1 } # as_fn_exit # as_fn_unset VAR # --------------- # Portably unset VAR. as_fn_unset () { { eval $1=; unset $1;} } as_unset=as_fn_unset # as_fn_append VAR VALUE # ---------------------- # Append the text in VALUE to the end of the definition contained in VAR. Take # advantage of any shell optimizations that allow amortized linear growth over # repeated appends, instead of the typical quadratic growth present in naive # implementations. if (eval "as_var=1; as_var+=2; test x\$as_var = x12") 2>/dev/null; then : eval 'as_fn_append () { eval $1+=\$2 }' else as_fn_append () { eval $1=\$$1\$2 } fi # as_fn_append # as_fn_arith ARG... # ------------------ # Perform arithmetic evaluation on the ARGs, and store the result in the # global $as_val. Take advantage of shells that can avoid forks. The arguments # must be portable across $(()) and expr. if (eval "test \$(( 1 + 1 )) = 2") 2>/dev/null; then : eval 'as_fn_arith () { as_val=$(( $* )) }' else as_fn_arith () { as_val=`expr "$@" || test $? -eq 1` } fi # as_fn_arith if expr a : '\(a\)' >/dev/null 2>&1 && test "X`expr 00001 : '.*\(...\)'`" = X001; then as_expr=expr else as_expr=false fi if (basename -- /) >/dev/null 2>&1 && test "X`basename -- / 2>&1`" = "X/"; then as_basename=basename else as_basename=false fi if (as_dir=`dirname -- /` && test "X$as_dir" = X/) >/dev/null 2>&1; then as_dirname=dirname else as_dirname=false fi as_me=`$as_basename -- "$0" || $as_expr X/"$0" : '.*/\([^/][^/]*\)/*$' \| \ X"$0" : 'X\(//\)$' \| \ X"$0" : 'X\(/\)' \| . 2>/dev/null || $as_echo X/"$0" | sed '/^.*\/\([^/][^/]*\)\/*$/{ s//\1/ q } /^X\/\(\/\/\)$/{ s//\1/ q } /^X\/\(\/\).*/{ s//\1/ q } s/.*/./; q'` # Avoid depending upon Character Ranges. as_cr_letters='abcdefghijklmnopqrstuvwxyz' as_cr_LETTERS='ABCDEFGHIJKLMNOPQRSTUVWXYZ' as_cr_Letters=$as_cr_letters$as_cr_LETTERS as_cr_digits='0123456789' as_cr_alnum=$as_cr_Letters$as_cr_digits ECHO_C= ECHO_N= ECHO_T= case `echo -n x` in #((((( -n*) case `echo 'xy\c'` in *c*) ECHO_T=' ';; # ECHO_T is single tab character. xy) ECHO_C='\c';; *) echo `echo ksh88 bug on AIX 6.1` > /dev/null ECHO_T=' ';; esac;; *) ECHO_N='-n';; esac rm -f conf$$ conf$$.exe conf$$.file if test -d conf$$.dir; then rm -f conf$$.dir/conf$$.file else rm -f conf$$.dir mkdir conf$$.dir 2>/dev/null fi if (echo >conf$$.file) 2>/dev/null; then if ln -s conf$$.file conf$$ 2>/dev/null; then as_ln_s='ln -s' # ... but there are two gotchas: # 1) On MSYS, both `ln -s file dir' and `ln file dir' fail. # 2) DJGPP < 2.04 has no symlinks; `ln -s' creates a wrapper executable. # In both cases, we have to default to `cp -pR'. ln -s conf$$.file conf$$.dir 2>/dev/null && test ! -f conf$$.exe || as_ln_s='cp -pR' elif ln conf$$.file conf$$ 2>/dev/null; then as_ln_s=ln else as_ln_s='cp -pR' fi else as_ln_s='cp -pR' fi rm -f conf$$ conf$$.exe conf$$.dir/conf$$.file conf$$.file rmdir conf$$.dir 2>/dev/null # as_fn_mkdir_p # ------------- # Create "$as_dir" as a directory, including parents if necessary. as_fn_mkdir_p () { case $as_dir in #( -*) as_dir=./$as_dir;; esac test -d "$as_dir" || eval $as_mkdir_p || { as_dirs= while :; do case $as_dir in #( *\'*) as_qdir=`$as_echo "$as_dir" | sed "s/'/'\\\\\\\\''/g"`;; #'( *) as_qdir=$as_dir;; esac as_dirs="'$as_qdir' $as_dirs" as_dir=`$as_dirname -- "$as_dir" || $as_expr X"$as_dir" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ X"$as_dir" : 'X\(//\)[^/]' \| \ X"$as_dir" : 'X\(//\)$' \| \ X"$as_dir" : 'X\(/\)' \| . 2>/dev/null || $as_echo X"$as_dir" | sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ s//\1/ q } /^X\(\/\/\)[^/].*/{ s//\1/ q } /^X\(\/\/\)$/{ s//\1/ q } /^X\(\/\).*/{ s//\1/ q } s/.*/./; q'` test -d "$as_dir" && break done test -z "$as_dirs" || eval "mkdir $as_dirs" } || test -d "$as_dir" || as_fn_error $? "cannot create directory $as_dir" } # as_fn_mkdir_p if mkdir -p . 2>/dev/null; then as_mkdir_p='mkdir -p "$as_dir"' else test -d ./-p && rmdir ./-p as_mkdir_p=false fi # as_fn_executable_p FILE # ----------------------- # Test if FILE is an executable regular file. as_fn_executable_p () { test -f "$1" && test -x "$1" } # as_fn_executable_p as_test_x='test -x' as_executable_p=as_fn_executable_p # Sed expression to map a string onto a valid CPP name. as_tr_cpp="eval sed 'y%*$as_cr_letters%P$as_cr_LETTERS%;s%[^_$as_cr_alnum]%_%g'" # Sed expression to map a string onto a valid variable name. as_tr_sh="eval sed 'y%*+%pp%;s%[^_$as_cr_alnum]%_%g'" exec 6>&1 ## ----------------------------------- ## ## Main body of $CONFIG_STATUS script. ## ## ----------------------------------- ## _ASEOF test $as_write_fail = 0 && chmod +x $CONFIG_STATUS || ac_write_fail=1 cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 # Save the log message, to keep $0 and so on meaningful, and to # report actual input values of CONFIG_FILES etc. instead of their # values after options handling. ac_log=" This file was extended by vowpal_wabbit $as_me 8.5.0, which was generated by GNU Autoconf 2.69. Invocation command line was CONFIG_FILES = $CONFIG_FILES CONFIG_HEADERS = $CONFIG_HEADERS CONFIG_LINKS = $CONFIG_LINKS CONFIG_COMMANDS = $CONFIG_COMMANDS $ $0 $@ on `(hostname || uname -n) 2>/dev/null | sed 1q` " _ACEOF case $ac_config_files in *" "*) set x $ac_config_files; shift; ac_config_files=$*;; esac case $ac_config_headers in *" "*) set x $ac_config_headers; shift; ac_config_headers=$*;; esac cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 # Files that config.status was made for. config_files="$ac_config_files" config_headers="$ac_config_headers" config_commands="$ac_config_commands" _ACEOF cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 ac_cs_usage="\ \`$as_me' instantiates files and other configuration actions from templates according to the current configuration. Unless the files and actions are specified as TAGs, all are instantiated by default. Usage: $0 [OPTION]... [TAG]... -h, --help print this help, then exit -V, --version print version number and configuration settings, then exit --config print configuration, then exit -q, --quiet, --silent do not print progress messages -d, --debug don't remove temporary files --recheck update $as_me by reconfiguring in the same conditions --file=FILE[:TEMPLATE] instantiate the configuration file FILE --header=FILE[:TEMPLATE] instantiate the configuration header FILE Configuration files: $config_files Configuration headers: $config_headers Configuration commands: $config_commands Report bugs to . vowpal_wabbit home page: ." _ACEOF cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`" ac_cs_version="\\ vowpal_wabbit config.status 8.5.0 configured by $0, generated by GNU Autoconf 2.69, with options \\"\$ac_cs_config\\" Copyright (C) 2012 Free Software Foundation, Inc. This config.status script is free software; the Free Software Foundation gives unlimited permission to copy, distribute and modify it." ac_pwd='$ac_pwd' srcdir='$srcdir' INSTALL='$INSTALL' MKDIR_P='$MKDIR_P' AWK='$AWK' test -n "\$AWK" || AWK=awk _ACEOF cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 # The default lists apply if the user does not specify any file. ac_need_defaults=: while test $# != 0 do case $1 in --*=?*) ac_option=`expr "X$1" : 'X\([^=]*\)='` ac_optarg=`expr "X$1" : 'X[^=]*=\(.*\)'` ac_shift=: ;; --*=) ac_option=`expr "X$1" : 'X\([^=]*\)='` ac_optarg= ac_shift=: ;; *) ac_option=$1 ac_optarg=$2 ac_shift=shift ;; esac case $ac_option in # Handling of the options. -recheck | --recheck | --rechec | --reche | --rech | --rec | --re | --r) ac_cs_recheck=: ;; --version | --versio | --versi | --vers | --ver | --ve | --v | -V ) $as_echo "$ac_cs_version"; exit ;; --config | --confi | --conf | --con | --co | --c ) $as_echo "$ac_cs_config"; exit ;; --debug | --debu | --deb | --de | --d | -d ) debug=: ;; --file | --fil | --fi | --f ) $ac_shift case $ac_optarg in *\'*) ac_optarg=`$as_echo "$ac_optarg" | sed "s/'/'\\\\\\\\''/g"` ;; '') as_fn_error $? "missing file argument" ;; esac as_fn_append CONFIG_FILES " '$ac_optarg'" ac_need_defaults=false;; --header | --heade | --head | --hea ) $ac_shift case $ac_optarg in *\'*) ac_optarg=`$as_echo "$ac_optarg" | sed "s/'/'\\\\\\\\''/g"` ;; esac as_fn_append CONFIG_HEADERS " '$ac_optarg'" ac_need_defaults=false;; --he | --h) # Conflict between --help and --header as_fn_error $? "ambiguous option: \`$1' Try \`$0 --help' for more information.";; --help | --hel | -h ) $as_echo "$ac_cs_usage"; exit ;; -q | -quiet | --quiet | --quie | --qui | --qu | --q \ | -silent | --silent | --silen | --sile | --sil | --si | --s) ac_cs_silent=: ;; # This is an error. -*) as_fn_error $? "unrecognized option: \`$1' Try \`$0 --help' for more information." ;; *) as_fn_append ac_config_targets " $1" ac_need_defaults=false ;; esac shift done ac_configure_extra_args= if $ac_cs_silent; then exec 6>/dev/null ac_configure_extra_args="$ac_configure_extra_args --silent" fi _ACEOF cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 if \$ac_cs_recheck; then set X $SHELL '$0' $ac_configure_args \$ac_configure_extra_args --no-create --no-recursion shift \$as_echo "running CONFIG_SHELL=$SHELL \$*" >&6 CONFIG_SHELL='$SHELL' export CONFIG_SHELL exec "\$@" fi _ACEOF cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 exec 5>>config.log { echo sed 'h;s/./-/g;s/^.../## /;s/...$/ ##/;p;x;p;x' <<_ASBOX ## Running $as_me. ## _ASBOX $as_echo "$ac_log" } >&5 _ACEOF cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 # # INIT-COMMANDS # AMDEP_TRUE="$AMDEP_TRUE" ac_aux_dir="$ac_aux_dir" # The HP-UX ksh and POSIX shell print the target directory to stdout # if CDPATH is set. (unset CDPATH) >/dev/null 2>&1 && unset CDPATH sed_quote_subst='$sed_quote_subst' double_quote_subst='$double_quote_subst' delay_variable_subst='$delay_variable_subst' macro_version='`$ECHO "$macro_version" | $SED "$delay_single_quote_subst"`' macro_revision='`$ECHO "$macro_revision" | $SED "$delay_single_quote_subst"`' enable_shared='`$ECHO "$enable_shared" | $SED "$delay_single_quote_subst"`' enable_static='`$ECHO "$enable_static" | $SED "$delay_single_quote_subst"`' pic_mode='`$ECHO "$pic_mode" | $SED "$delay_single_quote_subst"`' enable_fast_install='`$ECHO "$enable_fast_install" | $SED "$delay_single_quote_subst"`' shared_archive_member_spec='`$ECHO "$shared_archive_member_spec" | $SED "$delay_single_quote_subst"`' SHELL='`$ECHO "$SHELL" | $SED "$delay_single_quote_subst"`' ECHO='`$ECHO "$ECHO" | $SED "$delay_single_quote_subst"`' PATH_SEPARATOR='`$ECHO "$PATH_SEPARATOR" | $SED "$delay_single_quote_subst"`' host_alias='`$ECHO "$host_alias" | $SED "$delay_single_quote_subst"`' host='`$ECHO "$host" | $SED "$delay_single_quote_subst"`' host_os='`$ECHO "$host_os" | $SED "$delay_single_quote_subst"`' build_alias='`$ECHO "$build_alias" | $SED "$delay_single_quote_subst"`' build='`$ECHO "$build" | $SED "$delay_single_quote_subst"`' build_os='`$ECHO "$build_os" | $SED "$delay_single_quote_subst"`' SED='`$ECHO "$SED" | $SED "$delay_single_quote_subst"`' Xsed='`$ECHO "$Xsed" | $SED "$delay_single_quote_subst"`' GREP='`$ECHO "$GREP" | $SED "$delay_single_quote_subst"`' EGREP='`$ECHO "$EGREP" | $SED "$delay_single_quote_subst"`' FGREP='`$ECHO "$FGREP" | $SED "$delay_single_quote_subst"`' LD='`$ECHO "$LD" | $SED "$delay_single_quote_subst"`' NM='`$ECHO "$NM" | $SED "$delay_single_quote_subst"`' LN_S='`$ECHO "$LN_S" | $SED "$delay_single_quote_subst"`' max_cmd_len='`$ECHO "$max_cmd_len" | $SED "$delay_single_quote_subst"`' ac_objext='`$ECHO "$ac_objext" | $SED "$delay_single_quote_subst"`' exeext='`$ECHO "$exeext" | $SED "$delay_single_quote_subst"`' lt_unset='`$ECHO "$lt_unset" | $SED "$delay_single_quote_subst"`' lt_SP2NL='`$ECHO "$lt_SP2NL" | $SED "$delay_single_quote_subst"`' lt_NL2SP='`$ECHO "$lt_NL2SP" | $SED "$delay_single_quote_subst"`' lt_cv_to_host_file_cmd='`$ECHO "$lt_cv_to_host_file_cmd" | $SED "$delay_single_quote_subst"`' lt_cv_to_tool_file_cmd='`$ECHO "$lt_cv_to_tool_file_cmd" | $SED "$delay_single_quote_subst"`' reload_flag='`$ECHO "$reload_flag" | $SED "$delay_single_quote_subst"`' reload_cmds='`$ECHO "$reload_cmds" | $SED "$delay_single_quote_subst"`' OBJDUMP='`$ECHO "$OBJDUMP" | $SED "$delay_single_quote_subst"`' deplibs_check_method='`$ECHO "$deplibs_check_method" | $SED "$delay_single_quote_subst"`' file_magic_cmd='`$ECHO "$file_magic_cmd" | $SED "$delay_single_quote_subst"`' file_magic_glob='`$ECHO "$file_magic_glob" | $SED "$delay_single_quote_subst"`' want_nocaseglob='`$ECHO "$want_nocaseglob" | $SED "$delay_single_quote_subst"`' DLLTOOL='`$ECHO "$DLLTOOL" | $SED "$delay_single_quote_subst"`' sharedlib_from_linklib_cmd='`$ECHO "$sharedlib_from_linklib_cmd" | $SED "$delay_single_quote_subst"`' AR='`$ECHO "$AR" | $SED "$delay_single_quote_subst"`' AR_FLAGS='`$ECHO "$AR_FLAGS" | $SED "$delay_single_quote_subst"`' archiver_list_spec='`$ECHO "$archiver_list_spec" | $SED "$delay_single_quote_subst"`' STRIP='`$ECHO "$STRIP" | $SED "$delay_single_quote_subst"`' RANLIB='`$ECHO "$RANLIB" | $SED "$delay_single_quote_subst"`' old_postinstall_cmds='`$ECHO "$old_postinstall_cmds" | $SED "$delay_single_quote_subst"`' old_postuninstall_cmds='`$ECHO "$old_postuninstall_cmds" | $SED "$delay_single_quote_subst"`' old_archive_cmds='`$ECHO "$old_archive_cmds" | $SED "$delay_single_quote_subst"`' lock_old_archive_extraction='`$ECHO "$lock_old_archive_extraction" | $SED "$delay_single_quote_subst"`' CC='`$ECHO "$CC" | $SED "$delay_single_quote_subst"`' CFLAGS='`$ECHO "$CFLAGS" | $SED "$delay_single_quote_subst"`' compiler='`$ECHO "$compiler" | $SED "$delay_single_quote_subst"`' GCC='`$ECHO "$GCC" | $SED "$delay_single_quote_subst"`' lt_cv_sys_global_symbol_pipe='`$ECHO "$lt_cv_sys_global_symbol_pipe" | $SED "$delay_single_quote_subst"`' lt_cv_sys_global_symbol_to_cdecl='`$ECHO "$lt_cv_sys_global_symbol_to_cdecl" | $SED "$delay_single_quote_subst"`' lt_cv_sys_global_symbol_to_import='`$ECHO "$lt_cv_sys_global_symbol_to_import" | $SED "$delay_single_quote_subst"`' lt_cv_sys_global_symbol_to_c_name_address='`$ECHO "$lt_cv_sys_global_symbol_to_c_name_address" | $SED "$delay_single_quote_subst"`' lt_cv_sys_global_symbol_to_c_name_address_lib_prefix='`$ECHO "$lt_cv_sys_global_symbol_to_c_name_address_lib_prefix" | $SED "$delay_single_quote_subst"`' lt_cv_nm_interface='`$ECHO "$lt_cv_nm_interface" | $SED "$delay_single_quote_subst"`' nm_file_list_spec='`$ECHO "$nm_file_list_spec" | $SED "$delay_single_quote_subst"`' lt_sysroot='`$ECHO "$lt_sysroot" | $SED "$delay_single_quote_subst"`' lt_cv_truncate_bin='`$ECHO "$lt_cv_truncate_bin" | $SED "$delay_single_quote_subst"`' objdir='`$ECHO "$objdir" | $SED "$delay_single_quote_subst"`' MAGIC_CMD='`$ECHO "$MAGIC_CMD" | $SED "$delay_single_quote_subst"`' lt_prog_compiler_no_builtin_flag='`$ECHO "$lt_prog_compiler_no_builtin_flag" | $SED "$delay_single_quote_subst"`' lt_prog_compiler_pic='`$ECHO "$lt_prog_compiler_pic" | $SED "$delay_single_quote_subst"`' lt_prog_compiler_wl='`$ECHO "$lt_prog_compiler_wl" | $SED "$delay_single_quote_subst"`' lt_prog_compiler_static='`$ECHO "$lt_prog_compiler_static" | $SED "$delay_single_quote_subst"`' lt_cv_prog_compiler_c_o='`$ECHO "$lt_cv_prog_compiler_c_o" | $SED "$delay_single_quote_subst"`' need_locks='`$ECHO "$need_locks" | $SED "$delay_single_quote_subst"`' MANIFEST_TOOL='`$ECHO "$MANIFEST_TOOL" | $SED "$delay_single_quote_subst"`' DSYMUTIL='`$ECHO "$DSYMUTIL" | $SED "$delay_single_quote_subst"`' NMEDIT='`$ECHO "$NMEDIT" | $SED "$delay_single_quote_subst"`' LIPO='`$ECHO "$LIPO" | $SED "$delay_single_quote_subst"`' OTOOL='`$ECHO "$OTOOL" | $SED "$delay_single_quote_subst"`' OTOOL64='`$ECHO "$OTOOL64" | $SED "$delay_single_quote_subst"`' libext='`$ECHO "$libext" | $SED "$delay_single_quote_subst"`' shrext_cmds='`$ECHO "$shrext_cmds" | $SED "$delay_single_quote_subst"`' extract_expsyms_cmds='`$ECHO "$extract_expsyms_cmds" | $SED "$delay_single_quote_subst"`' archive_cmds_need_lc='`$ECHO "$archive_cmds_need_lc" | $SED "$delay_single_quote_subst"`' enable_shared_with_static_runtimes='`$ECHO "$enable_shared_with_static_runtimes" | $SED "$delay_single_quote_subst"`' export_dynamic_flag_spec='`$ECHO "$export_dynamic_flag_spec" | $SED "$delay_single_quote_subst"`' whole_archive_flag_spec='`$ECHO "$whole_archive_flag_spec" | $SED "$delay_single_quote_subst"`' compiler_needs_object='`$ECHO "$compiler_needs_object" | $SED "$delay_single_quote_subst"`' old_archive_from_new_cmds='`$ECHO "$old_archive_from_new_cmds" | $SED "$delay_single_quote_subst"`' old_archive_from_expsyms_cmds='`$ECHO "$old_archive_from_expsyms_cmds" | $SED "$delay_single_quote_subst"`' archive_cmds='`$ECHO "$archive_cmds" | $SED "$delay_single_quote_subst"`' archive_expsym_cmds='`$ECHO "$archive_expsym_cmds" | $SED "$delay_single_quote_subst"`' module_cmds='`$ECHO "$module_cmds" | $SED "$delay_single_quote_subst"`' module_expsym_cmds='`$ECHO "$module_expsym_cmds" | $SED "$delay_single_quote_subst"`' with_gnu_ld='`$ECHO "$with_gnu_ld" | $SED "$delay_single_quote_subst"`' allow_undefined_flag='`$ECHO "$allow_undefined_flag" | $SED "$delay_single_quote_subst"`' no_undefined_flag='`$ECHO "$no_undefined_flag" | $SED "$delay_single_quote_subst"`' hardcode_libdir_flag_spec='`$ECHO "$hardcode_libdir_flag_spec" | $SED "$delay_single_quote_subst"`' hardcode_libdir_separator='`$ECHO "$hardcode_libdir_separator" | $SED "$delay_single_quote_subst"`' hardcode_direct='`$ECHO "$hardcode_direct" | $SED "$delay_single_quote_subst"`' hardcode_direct_absolute='`$ECHO "$hardcode_direct_absolute" | $SED "$delay_single_quote_subst"`' hardcode_minus_L='`$ECHO "$hardcode_minus_L" | $SED "$delay_single_quote_subst"`' hardcode_shlibpath_var='`$ECHO "$hardcode_shlibpath_var" | $SED "$delay_single_quote_subst"`' hardcode_automatic='`$ECHO "$hardcode_automatic" | $SED "$delay_single_quote_subst"`' inherit_rpath='`$ECHO "$inherit_rpath" | $SED "$delay_single_quote_subst"`' link_all_deplibs='`$ECHO "$link_all_deplibs" | $SED "$delay_single_quote_subst"`' always_export_symbols='`$ECHO "$always_export_symbols" | $SED "$delay_single_quote_subst"`' export_symbols_cmds='`$ECHO "$export_symbols_cmds" | $SED "$delay_single_quote_subst"`' exclude_expsyms='`$ECHO "$exclude_expsyms" | $SED "$delay_single_quote_subst"`' include_expsyms='`$ECHO "$include_expsyms" | $SED "$delay_single_quote_subst"`' prelink_cmds='`$ECHO "$prelink_cmds" | $SED "$delay_single_quote_subst"`' postlink_cmds='`$ECHO "$postlink_cmds" | $SED "$delay_single_quote_subst"`' file_list_spec='`$ECHO "$file_list_spec" | $SED "$delay_single_quote_subst"`' variables_saved_for_relink='`$ECHO "$variables_saved_for_relink" | $SED "$delay_single_quote_subst"`' need_lib_prefix='`$ECHO "$need_lib_prefix" | $SED "$delay_single_quote_subst"`' need_version='`$ECHO "$need_version" | $SED "$delay_single_quote_subst"`' version_type='`$ECHO "$version_type" | $SED "$delay_single_quote_subst"`' runpath_var='`$ECHO "$runpath_var" | $SED "$delay_single_quote_subst"`' shlibpath_var='`$ECHO "$shlibpath_var" | $SED "$delay_single_quote_subst"`' shlibpath_overrides_runpath='`$ECHO "$shlibpath_overrides_runpath" | $SED "$delay_single_quote_subst"`' libname_spec='`$ECHO "$libname_spec" | $SED "$delay_single_quote_subst"`' library_names_spec='`$ECHO "$library_names_spec" | $SED "$delay_single_quote_subst"`' soname_spec='`$ECHO "$soname_spec" | $SED "$delay_single_quote_subst"`' install_override_mode='`$ECHO "$install_override_mode" | $SED "$delay_single_quote_subst"`' postinstall_cmds='`$ECHO "$postinstall_cmds" | $SED "$delay_single_quote_subst"`' postuninstall_cmds='`$ECHO "$postuninstall_cmds" | $SED "$delay_single_quote_subst"`' finish_cmds='`$ECHO "$finish_cmds" | $SED "$delay_single_quote_subst"`' finish_eval='`$ECHO "$finish_eval" | $SED "$delay_single_quote_subst"`' hardcode_into_libs='`$ECHO "$hardcode_into_libs" | $SED "$delay_single_quote_subst"`' sys_lib_search_path_spec='`$ECHO "$sys_lib_search_path_spec" | $SED "$delay_single_quote_subst"`' configure_time_dlsearch_path='`$ECHO "$configure_time_dlsearch_path" | $SED "$delay_single_quote_subst"`' configure_time_lt_sys_library_path='`$ECHO "$configure_time_lt_sys_library_path" | $SED "$delay_single_quote_subst"`' hardcode_action='`$ECHO "$hardcode_action" | $SED "$delay_single_quote_subst"`' enable_dlopen='`$ECHO "$enable_dlopen" | $SED "$delay_single_quote_subst"`' enable_dlopen_self='`$ECHO "$enable_dlopen_self" | $SED "$delay_single_quote_subst"`' enable_dlopen_self_static='`$ECHO "$enable_dlopen_self_static" | $SED "$delay_single_quote_subst"`' old_striplib='`$ECHO "$old_striplib" | $SED "$delay_single_quote_subst"`' striplib='`$ECHO "$striplib" | $SED "$delay_single_quote_subst"`' compiler_lib_search_dirs='`$ECHO "$compiler_lib_search_dirs" | $SED "$delay_single_quote_subst"`' predep_objects='`$ECHO "$predep_objects" | $SED "$delay_single_quote_subst"`' postdep_objects='`$ECHO "$postdep_objects" | $SED "$delay_single_quote_subst"`' predeps='`$ECHO "$predeps" | $SED "$delay_single_quote_subst"`' postdeps='`$ECHO "$postdeps" | $SED "$delay_single_quote_subst"`' compiler_lib_search_path='`$ECHO "$compiler_lib_search_path" | $SED "$delay_single_quote_subst"`' LD_CXX='`$ECHO "$LD_CXX" | $SED "$delay_single_quote_subst"`' reload_flag_CXX='`$ECHO "$reload_flag_CXX" | $SED "$delay_single_quote_subst"`' reload_cmds_CXX='`$ECHO "$reload_cmds_CXX" | $SED "$delay_single_quote_subst"`' old_archive_cmds_CXX='`$ECHO "$old_archive_cmds_CXX" | $SED "$delay_single_quote_subst"`' compiler_CXX='`$ECHO "$compiler_CXX" | $SED "$delay_single_quote_subst"`' GCC_CXX='`$ECHO "$GCC_CXX" | $SED "$delay_single_quote_subst"`' lt_prog_compiler_no_builtin_flag_CXX='`$ECHO "$lt_prog_compiler_no_builtin_flag_CXX" | $SED "$delay_single_quote_subst"`' lt_prog_compiler_pic_CXX='`$ECHO "$lt_prog_compiler_pic_CXX" | $SED "$delay_single_quote_subst"`' lt_prog_compiler_wl_CXX='`$ECHO "$lt_prog_compiler_wl_CXX" | $SED "$delay_single_quote_subst"`' lt_prog_compiler_static_CXX='`$ECHO "$lt_prog_compiler_static_CXX" | $SED "$delay_single_quote_subst"`' lt_cv_prog_compiler_c_o_CXX='`$ECHO "$lt_cv_prog_compiler_c_o_CXX" | $SED "$delay_single_quote_subst"`' archive_cmds_need_lc_CXX='`$ECHO "$archive_cmds_need_lc_CXX" | $SED "$delay_single_quote_subst"`' enable_shared_with_static_runtimes_CXX='`$ECHO "$enable_shared_with_static_runtimes_CXX" | $SED "$delay_single_quote_subst"`' export_dynamic_flag_spec_CXX='`$ECHO "$export_dynamic_flag_spec_CXX" | $SED "$delay_single_quote_subst"`' whole_archive_flag_spec_CXX='`$ECHO "$whole_archive_flag_spec_CXX" | $SED "$delay_single_quote_subst"`' compiler_needs_object_CXX='`$ECHO "$compiler_needs_object_CXX" | $SED "$delay_single_quote_subst"`' old_archive_from_new_cmds_CXX='`$ECHO "$old_archive_from_new_cmds_CXX" | $SED "$delay_single_quote_subst"`' old_archive_from_expsyms_cmds_CXX='`$ECHO "$old_archive_from_expsyms_cmds_CXX" | $SED "$delay_single_quote_subst"`' archive_cmds_CXX='`$ECHO "$archive_cmds_CXX" | $SED "$delay_single_quote_subst"`' archive_expsym_cmds_CXX='`$ECHO "$archive_expsym_cmds_CXX" | $SED "$delay_single_quote_subst"`' module_cmds_CXX='`$ECHO "$module_cmds_CXX" | $SED "$delay_single_quote_subst"`' module_expsym_cmds_CXX='`$ECHO "$module_expsym_cmds_CXX" | $SED "$delay_single_quote_subst"`' with_gnu_ld_CXX='`$ECHO "$with_gnu_ld_CXX" | $SED "$delay_single_quote_subst"`' allow_undefined_flag_CXX='`$ECHO "$allow_undefined_flag_CXX" | $SED "$delay_single_quote_subst"`' no_undefined_flag_CXX='`$ECHO "$no_undefined_flag_CXX" | $SED "$delay_single_quote_subst"`' hardcode_libdir_flag_spec_CXX='`$ECHO "$hardcode_libdir_flag_spec_CXX" | $SED "$delay_single_quote_subst"`' hardcode_libdir_separator_CXX='`$ECHO "$hardcode_libdir_separator_CXX" | $SED "$delay_single_quote_subst"`' hardcode_direct_CXX='`$ECHO "$hardcode_direct_CXX" | $SED "$delay_single_quote_subst"`' hardcode_direct_absolute_CXX='`$ECHO "$hardcode_direct_absolute_CXX" | $SED "$delay_single_quote_subst"`' hardcode_minus_L_CXX='`$ECHO "$hardcode_minus_L_CXX" | $SED "$delay_single_quote_subst"`' hardcode_shlibpath_var_CXX='`$ECHO "$hardcode_shlibpath_var_CXX" | $SED "$delay_single_quote_subst"`' hardcode_automatic_CXX='`$ECHO "$hardcode_automatic_CXX" | $SED "$delay_single_quote_subst"`' inherit_rpath_CXX='`$ECHO "$inherit_rpath_CXX" | $SED "$delay_single_quote_subst"`' link_all_deplibs_CXX='`$ECHO "$link_all_deplibs_CXX" | $SED "$delay_single_quote_subst"`' always_export_symbols_CXX='`$ECHO "$always_export_symbols_CXX" | $SED "$delay_single_quote_subst"`' export_symbols_cmds_CXX='`$ECHO "$export_symbols_cmds_CXX" | $SED "$delay_single_quote_subst"`' exclude_expsyms_CXX='`$ECHO "$exclude_expsyms_CXX" | $SED "$delay_single_quote_subst"`' include_expsyms_CXX='`$ECHO "$include_expsyms_CXX" | $SED "$delay_single_quote_subst"`' prelink_cmds_CXX='`$ECHO "$prelink_cmds_CXX" | $SED "$delay_single_quote_subst"`' postlink_cmds_CXX='`$ECHO "$postlink_cmds_CXX" | $SED "$delay_single_quote_subst"`' file_list_spec_CXX='`$ECHO "$file_list_spec_CXX" | $SED "$delay_single_quote_subst"`' hardcode_action_CXX='`$ECHO "$hardcode_action_CXX" | $SED "$delay_single_quote_subst"`' compiler_lib_search_dirs_CXX='`$ECHO "$compiler_lib_search_dirs_CXX" | $SED "$delay_single_quote_subst"`' predep_objects_CXX='`$ECHO "$predep_objects_CXX" | $SED "$delay_single_quote_subst"`' postdep_objects_CXX='`$ECHO "$postdep_objects_CXX" | $SED "$delay_single_quote_subst"`' predeps_CXX='`$ECHO "$predeps_CXX" | $SED "$delay_single_quote_subst"`' postdeps_CXX='`$ECHO "$postdeps_CXX" | $SED "$delay_single_quote_subst"`' compiler_lib_search_path_CXX='`$ECHO "$compiler_lib_search_path_CXX" | $SED "$delay_single_quote_subst"`' LTCC='$LTCC' LTCFLAGS='$LTCFLAGS' compiler='$compiler_DEFAULT' # A function that is used when there is no print builtin or printf. func_fallback_echo () { eval 'cat <<_LTECHO_EOF \$1 _LTECHO_EOF' } # Quote evaled strings. for var in SHELL \ ECHO \ PATH_SEPARATOR \ SED \ GREP \ EGREP \ FGREP \ LD \ NM \ LN_S \ lt_SP2NL \ lt_NL2SP \ reload_flag \ OBJDUMP \ deplibs_check_method \ file_magic_cmd \ file_magic_glob \ want_nocaseglob \ DLLTOOL \ sharedlib_from_linklib_cmd \ AR \ AR_FLAGS \ archiver_list_spec \ STRIP \ RANLIB \ CC \ CFLAGS \ compiler \ lt_cv_sys_global_symbol_pipe \ lt_cv_sys_global_symbol_to_cdecl \ lt_cv_sys_global_symbol_to_import \ lt_cv_sys_global_symbol_to_c_name_address \ lt_cv_sys_global_symbol_to_c_name_address_lib_prefix \ lt_cv_nm_interface \ nm_file_list_spec \ lt_cv_truncate_bin \ lt_prog_compiler_no_builtin_flag \ lt_prog_compiler_pic \ lt_prog_compiler_wl \ lt_prog_compiler_static \ lt_cv_prog_compiler_c_o \ need_locks \ MANIFEST_TOOL \ DSYMUTIL \ NMEDIT \ LIPO \ OTOOL \ OTOOL64 \ shrext_cmds \ export_dynamic_flag_spec \ whole_archive_flag_spec \ compiler_needs_object \ with_gnu_ld \ allow_undefined_flag \ no_undefined_flag \ hardcode_libdir_flag_spec \ hardcode_libdir_separator \ exclude_expsyms \ include_expsyms \ file_list_spec \ variables_saved_for_relink \ libname_spec \ library_names_spec \ soname_spec \ install_override_mode \ finish_eval \ old_striplib \ striplib \ compiler_lib_search_dirs \ predep_objects \ postdep_objects \ predeps \ postdeps \ compiler_lib_search_path \ LD_CXX \ reload_flag_CXX \ compiler_CXX \ lt_prog_compiler_no_builtin_flag_CXX \ lt_prog_compiler_pic_CXX \ lt_prog_compiler_wl_CXX \ lt_prog_compiler_static_CXX \ lt_cv_prog_compiler_c_o_CXX \ export_dynamic_flag_spec_CXX \ whole_archive_flag_spec_CXX \ compiler_needs_object_CXX \ with_gnu_ld_CXX \ allow_undefined_flag_CXX \ no_undefined_flag_CXX \ hardcode_libdir_flag_spec_CXX \ hardcode_libdir_separator_CXX \ exclude_expsyms_CXX \ include_expsyms_CXX \ file_list_spec_CXX \ compiler_lib_search_dirs_CXX \ predep_objects_CXX \ postdep_objects_CXX \ predeps_CXX \ postdeps_CXX \ compiler_lib_search_path_CXX; do case \`eval \\\\\$ECHO \\\\""\\\\\$\$var"\\\\"\` in *[\\\\\\\`\\"\\\$]*) eval "lt_\$var=\\\\\\"\\\`\\\$ECHO \\"\\\$\$var\\" | \\\$SED \\"\\\$sed_quote_subst\\"\\\`\\\\\\"" ## exclude from sc_prohibit_nested_quotes ;; *) eval "lt_\$var=\\\\\\"\\\$\$var\\\\\\"" ;; esac done # Double-quote double-evaled strings. for var in reload_cmds \ old_postinstall_cmds \ old_postuninstall_cmds \ old_archive_cmds \ extract_expsyms_cmds \ old_archive_from_new_cmds \ old_archive_from_expsyms_cmds \ archive_cmds \ archive_expsym_cmds \ module_cmds \ module_expsym_cmds \ export_symbols_cmds \ prelink_cmds \ postlink_cmds \ postinstall_cmds \ postuninstall_cmds \ finish_cmds \ sys_lib_search_path_spec \ configure_time_dlsearch_path \ configure_time_lt_sys_library_path \ reload_cmds_CXX \ old_archive_cmds_CXX \ old_archive_from_new_cmds_CXX \ old_archive_from_expsyms_cmds_CXX \ archive_cmds_CXX \ archive_expsym_cmds_CXX \ module_cmds_CXX \ module_expsym_cmds_CXX \ export_symbols_cmds_CXX \ prelink_cmds_CXX \ postlink_cmds_CXX; do case \`eval \\\\\$ECHO \\\\""\\\\\$\$var"\\\\"\` in *[\\\\\\\`\\"\\\$]*) eval "lt_\$var=\\\\\\"\\\`\\\$ECHO \\"\\\$\$var\\" | \\\$SED -e \\"\\\$double_quote_subst\\" -e \\"\\\$sed_quote_subst\\" -e \\"\\\$delay_variable_subst\\"\\\`\\\\\\"" ## exclude from sc_prohibit_nested_quotes ;; *) eval "lt_\$var=\\\\\\"\\\$\$var\\\\\\"" ;; esac done ac_aux_dir='$ac_aux_dir' # See if we are running on zsh, and set the options that allow our # commands through without removal of \ escapes INIT. if test -n "\${ZSH_VERSION+set}"; then setopt NO_GLOB_SUBST fi PACKAGE='$PACKAGE' VERSION='$VERSION' RM='$RM' ofile='$ofile' _ACEOF cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 # Handling of arguments. for ac_config_target in $ac_config_targets do case $ac_config_target in "vowpalwabbit/config.h") CONFIG_HEADERS="$CONFIG_HEADERS vowpalwabbit/config.h" ;; "depfiles") CONFIG_COMMANDS="$CONFIG_COMMANDS depfiles" ;; "libtool") CONFIG_COMMANDS="$CONFIG_COMMANDS libtool" ;; "Makefile") CONFIG_FILES="$CONFIG_FILES Makefile" ;; "vowpalwabbit/Makefile") CONFIG_FILES="$CONFIG_FILES vowpalwabbit/Makefile" ;; "cluster/Makefile") CONFIG_FILES="$CONFIG_FILES cluster/Makefile" ;; "library/Makefile") CONFIG_FILES="$CONFIG_FILES library/Makefile" ;; "libvw.pc") CONFIG_FILES="$CONFIG_FILES libvw.pc" ;; "libvw_c_wrapper.pc") CONFIG_FILES="$CONFIG_FILES libvw_c_wrapper.pc" ;; *) as_fn_error $? "invalid argument: \`$ac_config_target'" "$LINENO" 5;; esac done # If the user did not use the arguments to specify the items to instantiate, # then the envvar interface is used. Set only those that are not. # We use the long form for the default assignment because of an extremely # bizarre bug on SunOS 4.1.3. if $ac_need_defaults; then test "${CONFIG_FILES+set}" = set || CONFIG_FILES=$config_files test "${CONFIG_HEADERS+set}" = set || CONFIG_HEADERS=$config_headers test "${CONFIG_COMMANDS+set}" = set || CONFIG_COMMANDS=$config_commands fi # Have a temporary directory for convenience. Make it in the build tree # simply because there is no reason against having it here, and in addition, # creating and moving files from /tmp can sometimes cause problems. # Hook for its removal unless debugging. # Note that there is a small window in which the directory will not be cleaned: # after its creation but before its name has been assigned to `$tmp'. $debug || { tmp= ac_tmp= trap 'exit_status=$? : "${ac_tmp:=$tmp}" { test ! -d "$ac_tmp" || rm -fr "$ac_tmp"; } && exit $exit_status ' 0 trap 'as_fn_exit 1' 1 2 13 15 } # Create a (secure) tmp directory for tmp files. { tmp=`(umask 077 && mktemp -d "./confXXXXXX") 2>/dev/null` && test -d "$tmp" } || { tmp=./conf$$-$RANDOM (umask 077 && mkdir "$tmp") } || as_fn_error $? "cannot create a temporary directory in ." "$LINENO" 5 ac_tmp=$tmp # Set up the scripts for CONFIG_FILES section. # No need to generate them if there are no CONFIG_FILES. # This happens for instance with `./config.status config.h'. if test -n "$CONFIG_FILES"; then ac_cr=`echo X | tr X '\015'` # On cygwin, bash can eat \r inside `` if the user requested igncr. # But we know of no other shell where ac_cr would be empty at this # point, so we can use a bashism as a fallback. if test "x$ac_cr" = x; then eval ac_cr=\$\'\\r\' fi ac_cs_awk_cr=`$AWK 'BEGIN { print "a\rb" }' /dev/null` if test "$ac_cs_awk_cr" = "a${ac_cr}b"; then ac_cs_awk_cr='\\r' else ac_cs_awk_cr=$ac_cr fi echo 'BEGIN {' >"$ac_tmp/subs1.awk" && _ACEOF { echo "cat >conf$$subs.awk <<_ACEOF" && echo "$ac_subst_vars" | sed 's/.*/&!$&$ac_delim/' && echo "_ACEOF" } >conf$$subs.sh || as_fn_error $? "could not make $CONFIG_STATUS" "$LINENO" 5 ac_delim_num=`echo "$ac_subst_vars" | grep -c '^'` ac_delim='%!_!# ' for ac_last_try in false false false false false :; do . ./conf$$subs.sh || as_fn_error $? "could not make $CONFIG_STATUS" "$LINENO" 5 ac_delim_n=`sed -n "s/.*$ac_delim\$/X/p" conf$$subs.awk | grep -c X` if test $ac_delim_n = $ac_delim_num; then break elif $ac_last_try; then as_fn_error $? "could not make $CONFIG_STATUS" "$LINENO" 5 else ac_delim="$ac_delim!$ac_delim _$ac_delim!! " fi done rm -f conf$$subs.sh cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 cat >>"\$ac_tmp/subs1.awk" <<\\_ACAWK && _ACEOF sed -n ' h s/^/S["/; s/!.*/"]=/ p g s/^[^!]*!// :repl t repl s/'"$ac_delim"'$// t delim :nl h s/\(.\{148\}\)..*/\1/ t more1 s/["\\]/\\&/g; s/^/"/; s/$/\\n"\\/ p n b repl :more1 s/["\\]/\\&/g; s/^/"/; s/$/"\\/ p g s/.\{148\}// t nl :delim h s/\(.\{148\}\)..*/\1/ t more2 s/["\\]/\\&/g; s/^/"/; s/$/"/ p b :more2 s/["\\]/\\&/g; s/^/"/; s/$/"\\/ p g s/.\{148\}// t delim ' >$CONFIG_STATUS || ac_write_fail=1 rm -f conf$$subs.awk cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 _ACAWK cat >>"\$ac_tmp/subs1.awk" <<_ACAWK && for (key in S) S_is_set[key] = 1 FS = "" } { line = $ 0 nfields = split(line, field, "@") substed = 0 len = length(field[1]) for (i = 2; i < nfields; i++) { key = field[i] keylen = length(key) if (S_is_set[key]) { value = S[key] line = substr(line, 1, len) "" value "" substr(line, len + keylen + 3) len += length(value) + length(field[++i]) substed = 1 } else len += 1 + keylen } print line } _ACAWK _ACEOF cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 if sed "s/$ac_cr//" < /dev/null > /dev/null 2>&1; then sed "s/$ac_cr\$//; s/$ac_cr/$ac_cs_awk_cr/g" else cat fi < "$ac_tmp/subs1.awk" > "$ac_tmp/subs.awk" \ || as_fn_error $? "could not setup config files machinery" "$LINENO" 5 _ACEOF # VPATH may cause trouble with some makes, so we remove sole $(srcdir), # ${srcdir} and @srcdir@ entries from VPATH if srcdir is ".", strip leading and # trailing colons and then remove the whole line if VPATH becomes empty # (actually we leave an empty line to preserve line numbers). if test "x$srcdir" = x.; then ac_vpsub='/^[ ]*VPATH[ ]*=[ ]*/{ h s/// s/^/:/ s/[ ]*$/:/ s/:\$(srcdir):/:/g s/:\${srcdir}:/:/g s/:@srcdir@:/:/g s/^:*// s/:*$// x s/\(=[ ]*\).*/\1/ G s/\n// s/^[^=]*=[ ]*$// }' fi cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 fi # test -n "$CONFIG_FILES" # Set up the scripts for CONFIG_HEADERS section. # No need to generate them if there are no CONFIG_HEADERS. # This happens for instance with `./config.status Makefile'. if test -n "$CONFIG_HEADERS"; then cat >"$ac_tmp/defines.awk" <<\_ACAWK || BEGIN { _ACEOF # Transform confdefs.h into an awk script `defines.awk', embedded as # here-document in config.status, that substitutes the proper values into # config.h.in to produce config.h. # Create a delimiter string that does not exist in confdefs.h, to ease # handling of long lines. ac_delim='%!_!# ' for ac_last_try in false false :; do ac_tt=`sed -n "/$ac_delim/p" confdefs.h` if test -z "$ac_tt"; then break elif $ac_last_try; then as_fn_error $? "could not make $CONFIG_HEADERS" "$LINENO" 5 else ac_delim="$ac_delim!$ac_delim _$ac_delim!! " fi done # For the awk script, D is an array of macro values keyed by name, # likewise P contains macro parameters if any. Preserve backslash # newline sequences. ac_word_re=[_$as_cr_Letters][_$as_cr_alnum]* sed -n ' s/.\{148\}/&'"$ac_delim"'/g t rset :rset s/^[ ]*#[ ]*define[ ][ ]*/ / t def d :def s/\\$// t bsnl s/["\\]/\\&/g s/^ \('"$ac_word_re"'\)\(([^()]*)\)[ ]*\(.*\)/P["\1"]="\2"\ D["\1"]=" \3"/p s/^ \('"$ac_word_re"'\)[ ]*\(.*\)/D["\1"]=" \2"/p d :bsnl s/["\\]/\\&/g s/^ \('"$ac_word_re"'\)\(([^()]*)\)[ ]*\(.*\)/P["\1"]="\2"\ D["\1"]=" \3\\\\\\n"\\/p t cont s/^ \('"$ac_word_re"'\)[ ]*\(.*\)/D["\1"]=" \2\\\\\\n"\\/p t cont d :cont n s/.\{148\}/&'"$ac_delim"'/g t clear :clear s/\\$// t bsnlc s/["\\]/\\&/g; s/^/"/; s/$/"/p d :bsnlc s/["\\]/\\&/g; s/^/"/; s/$/\\\\\\n"\\/p b cont ' >$CONFIG_STATUS || ac_write_fail=1 cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 for (key in D) D_is_set[key] = 1 FS = "" } /^[\t ]*#[\t ]*(define|undef)[\t ]+$ac_word_re([\t (]|\$)/ { line = \$ 0 split(line, arg, " ") if (arg[1] == "#") { defundef = arg[2] mac1 = arg[3] } else { defundef = substr(arg[1], 2) mac1 = arg[2] } split(mac1, mac2, "(") #) macro = mac2[1] prefix = substr(line, 1, index(line, defundef) - 1) if (D_is_set[macro]) { # Preserve the white space surrounding the "#". print prefix "define", macro P[macro] D[macro] next } else { # Replace #undef with comments. This is necessary, for example, # in the case of _POSIX_SOURCE, which is predefined and required # on some systems where configure will not decide to define it. if (defundef == "undef") { print "/*", prefix defundef, macro, "*/" next } } } { print } _ACAWK _ACEOF cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 as_fn_error $? "could not setup config headers machinery" "$LINENO" 5 fi # test -n "$CONFIG_HEADERS" eval set X " :F $CONFIG_FILES :H $CONFIG_HEADERS :C $CONFIG_COMMANDS" shift for ac_tag do case $ac_tag in :[FHLC]) ac_mode=$ac_tag; continue;; esac case $ac_mode$ac_tag in :[FHL]*:*);; :L* | :C*:*) as_fn_error $? "invalid tag \`$ac_tag'" "$LINENO" 5;; :[FH]-) ac_tag=-:-;; :[FH]*) ac_tag=$ac_tag:$ac_tag.in;; esac ac_save_IFS=$IFS IFS=: set x $ac_tag IFS=$ac_save_IFS shift ac_file=$1 shift case $ac_mode in :L) ac_source=$1;; :[FH]) ac_file_inputs= for ac_f do case $ac_f in -) ac_f="$ac_tmp/stdin";; *) # Look for the file first in the build tree, then in the source tree # (if the path is not absolute). The absolute path cannot be DOS-style, # because $ac_f cannot contain `:'. test -f "$ac_f" || case $ac_f in [\\/$]*) false;; *) test -f "$srcdir/$ac_f" && ac_f="$srcdir/$ac_f";; esac || as_fn_error 1 "cannot find input file: \`$ac_f'" "$LINENO" 5;; esac case $ac_f in *\'*) ac_f=`$as_echo "$ac_f" | sed "s/'/'\\\\\\\\''/g"`;; esac as_fn_append ac_file_inputs " '$ac_f'" done # Let's still pretend it is `configure' which instantiates (i.e., don't # use $as_me), people would be surprised to read: # /* config.h. Generated by config.status. */ configure_input='Generated from '` $as_echo "$*" | sed 's|^[^:]*/||;s|:[^:]*/|, |g' `' by configure.' if test x"$ac_file" != x-; then configure_input="$ac_file. $configure_input" { $as_echo "$as_me:${as_lineno-$LINENO}: creating $ac_file" >&5 $as_echo "$as_me: creating $ac_file" >&6;} fi # Neutralize special characters interpreted by sed in replacement strings. case $configure_input in #( *\&* | *\|* | *\\* ) ac_sed_conf_input=`$as_echo "$configure_input" | sed 's/[\\\\&|]/\\\\&/g'`;; #( *) ac_sed_conf_input=$configure_input;; esac case $ac_tag in *:-:* | *:-) cat >"$ac_tmp/stdin" \ || as_fn_error $? "could not create $ac_file" "$LINENO" 5 ;; esac ;; esac ac_dir=`$as_dirname -- "$ac_file" || $as_expr X"$ac_file" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ X"$ac_file" : 'X\(//\)[^/]' \| \ X"$ac_file" : 'X\(//\)$' \| \ X"$ac_file" : 'X\(/\)' \| . 2>/dev/null || $as_echo X"$ac_file" | sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ s//\1/ q } /^X\(\/\/\)[^/].*/{ s//\1/ q } /^X\(\/\/\)$/{ s//\1/ q } /^X\(\/\).*/{ s//\1/ q } s/.*/./; q'` as_dir="$ac_dir"; as_fn_mkdir_p ac_builddir=. case "$ac_dir" in .) ac_dir_suffix= ac_top_builddir_sub=. ac_top_build_prefix= ;; *) ac_dir_suffix=/`$as_echo "$ac_dir" | sed 's|^\.[\\/]||'` # A ".." for each directory in $ac_dir_suffix. ac_top_builddir_sub=`$as_echo "$ac_dir_suffix" | sed 's|/[^\\/]*|/..|g;s|/||'` case $ac_top_builddir_sub in "") ac_top_builddir_sub=. ac_top_build_prefix= ;; *) ac_top_build_prefix=$ac_top_builddir_sub/ ;; esac ;; esac ac_abs_top_builddir=$ac_pwd ac_abs_builddir=$ac_pwd$ac_dir_suffix # for backward compatibility: ac_top_builddir=$ac_top_build_prefix case $srcdir in .) # We are building in place. ac_srcdir=. ac_top_srcdir=$ac_top_builddir_sub ac_abs_top_srcdir=$ac_pwd ;; [\\/]* | ?:[\\/]* ) # Absolute name. ac_srcdir=$srcdir$ac_dir_suffix; ac_top_srcdir=$srcdir ac_abs_top_srcdir=$srcdir ;; *) # Relative name. ac_srcdir=$ac_top_build_prefix$srcdir$ac_dir_suffix ac_top_srcdir=$ac_top_build_prefix$srcdir ac_abs_top_srcdir=$ac_pwd/$srcdir ;; esac ac_abs_srcdir=$ac_abs_top_srcdir$ac_dir_suffix case $ac_mode in :F) # # CONFIG_FILE # case $INSTALL in [\\/$]* | ?:[\\/]* ) ac_INSTALL=$INSTALL ;; *) ac_INSTALL=$ac_top_build_prefix$INSTALL ;; esac ac_MKDIR_P=$MKDIR_P case $MKDIR_P in [\\/$]* | ?:[\\/]* ) ;; */*) ac_MKDIR_P=$ac_top_build_prefix$MKDIR_P ;; esac _ACEOF cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 # If the template does not know about datarootdir, expand it. # FIXME: This hack should be removed a few years after 2.60. ac_datarootdir_hack=; ac_datarootdir_seen= ac_sed_dataroot=' /datarootdir/ { p q } /@datadir@/p /@docdir@/p /@infodir@/p /@localedir@/p /@mandir@/p' case `eval "sed -n \"\$ac_sed_dataroot\" $ac_file_inputs"` in *datarootdir*) ac_datarootdir_seen=yes;; *@datadir@*|*@docdir@*|*@infodir@*|*@localedir@*|*@mandir@*) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $ac_file_inputs seems to ignore the --datarootdir setting" >&5 $as_echo "$as_me: WARNING: $ac_file_inputs seems to ignore the --datarootdir setting" >&2;} _ACEOF cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 ac_datarootdir_hack=' s&@datadir@&$datadir&g s&@docdir@&$docdir&g s&@infodir@&$infodir&g s&@localedir@&$localedir&g s&@mandir@&$mandir&g s&\\\${datarootdir}&$datarootdir&g' ;; esac _ACEOF # Neutralize VPATH when `$srcdir' = `.'. # Shell code in configure.ac might set extrasub. # FIXME: do we really want to maintain this feature? cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 ac_sed_extra="$ac_vpsub $extrasub _ACEOF cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 :t /@[a-zA-Z_][a-zA-Z_0-9]*@/!b s|@configure_input@|$ac_sed_conf_input|;t t s&@top_builddir@&$ac_top_builddir_sub&;t t s&@top_build_prefix@&$ac_top_build_prefix&;t t s&@srcdir@&$ac_srcdir&;t t s&@abs_srcdir@&$ac_abs_srcdir&;t t s&@top_srcdir@&$ac_top_srcdir&;t t s&@abs_top_srcdir@&$ac_abs_top_srcdir&;t t s&@builddir@&$ac_builddir&;t t s&@abs_builddir@&$ac_abs_builddir&;t t s&@abs_top_builddir@&$ac_abs_top_builddir&;t t s&@INSTALL@&$ac_INSTALL&;t t s&@MKDIR_P@&$ac_MKDIR_P&;t t $ac_datarootdir_hack " eval sed \"\$ac_sed_extra\" "$ac_file_inputs" | $AWK -f "$ac_tmp/subs.awk" \ >$ac_tmp/out || as_fn_error $? "could not create $ac_file" "$LINENO" 5 test -z "$ac_datarootdir_hack$ac_datarootdir_seen" && { ac_out=`sed -n '/\${datarootdir}/p' "$ac_tmp/out"`; test -n "$ac_out"; } && { ac_out=`sed -n '/^[ ]*datarootdir[ ]*:*=/p' \ "$ac_tmp/out"`; test -z "$ac_out"; } && { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $ac_file contains a reference to the variable \`datarootdir' which seems to be undefined. Please make sure it is defined" >&5 $as_echo "$as_me: WARNING: $ac_file contains a reference to the variable \`datarootdir' which seems to be undefined. Please make sure it is defined" >&2;} rm -f "$ac_tmp/stdin" case $ac_file in -) cat "$ac_tmp/out" && rm -f "$ac_tmp/out";; *) rm -f "$ac_file" && mv "$ac_tmp/out" "$ac_file";; esac \ || as_fn_error $? "could not create $ac_file" "$LINENO" 5 ;; :H) # # CONFIG_HEADER # if test x"$ac_file" != x-; then { $as_echo "/* $configure_input */" \ && eval '$AWK -f "$ac_tmp/defines.awk"' "$ac_file_inputs" } >"$ac_tmp/config.h" \ || as_fn_error $? "could not create $ac_file" "$LINENO" 5 if diff "$ac_file" "$ac_tmp/config.h" >/dev/null 2>&1; then { $as_echo "$as_me:${as_lineno-$LINENO}: $ac_file is unchanged" >&5 $as_echo "$as_me: $ac_file is unchanged" >&6;} else rm -f "$ac_file" mv "$ac_tmp/config.h" "$ac_file" \ || as_fn_error $? "could not create $ac_file" "$LINENO" 5 fi else $as_echo "/* $configure_input */" \ && eval '$AWK -f "$ac_tmp/defines.awk"' "$ac_file_inputs" \ || as_fn_error $? "could not create -" "$LINENO" 5 fi # Compute "$ac_file"'s index in $config_headers. _am_arg="$ac_file" _am_stamp_count=1 for _am_header in $config_headers :; do case $_am_header in $_am_arg | $_am_arg:* ) break ;; * ) _am_stamp_count=`expr $_am_stamp_count + 1` ;; esac done echo "timestamp for $_am_arg" >`$as_dirname -- "$_am_arg" || $as_expr X"$_am_arg" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ X"$_am_arg" : 'X\(//\)[^/]' \| \ X"$_am_arg" : 'X\(//\)$' \| \ X"$_am_arg" : 'X\(/\)' \| . 2>/dev/null || $as_echo X"$_am_arg" | sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ s//\1/ q } /^X\(\/\/\)[^/].*/{ s//\1/ q } /^X\(\/\/\)$/{ s//\1/ q } /^X\(\/\).*/{ s//\1/ q } s/.*/./; q'`/stamp-h$_am_stamp_count ;; :C) { $as_echo "$as_me:${as_lineno-$LINENO}: executing $ac_file commands" >&5 $as_echo "$as_me: executing $ac_file commands" >&6;} ;; esac case $ac_file$ac_mode in "depfiles":C) test x"$AMDEP_TRUE" != x"" || { # Older Autoconf quotes --file arguments for eval, but not when files # are listed without --file. Let's play safe and only enable the eval # if we detect the quoting. case $CONFIG_FILES in *\'*) eval set x "$CONFIG_FILES" ;; *) set x $CONFIG_FILES ;; esac shift for mf do # Strip MF so we end up with the name of the file. mf=`echo "$mf" | sed -e 's/:.*$//'` # Check whether this is an Automake generated Makefile or not. # We used to match only the files named 'Makefile.in', but # some people rename them; so instead we look at the file content. # Grep'ing the first line is not enough: some people post-process # each Makefile.in and add a new line on top of each file to say so. # Grep'ing the whole file is not good either: AIX grep has a line # limit of 2048, but all sed's we know have understand at least 4000. if sed -n 's,^#.*generated by automake.*,X,p' "$mf" | grep X >/dev/null 2>&1; then dirpart=`$as_dirname -- "$mf" || $as_expr X"$mf" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ X"$mf" : 'X\(//\)[^/]' \| \ X"$mf" : 'X\(//\)$' \| \ X"$mf" : 'X\(/\)' \| . 2>/dev/null || $as_echo X"$mf" | sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ s//\1/ q } /^X\(\/\/\)[^/].*/{ s//\1/ q } /^X\(\/\/\)$/{ s//\1/ q } /^X\(\/\).*/{ s//\1/ q } s/.*/./; q'` else continue fi # Extract the definition of DEPDIR, am__include, and am__quote # from the Makefile without running 'make'. DEPDIR=`sed -n 's/^DEPDIR = //p' < "$mf"` test -z "$DEPDIR" && continue am__include=`sed -n 's/^am__include = //p' < "$mf"` test -z "$am__include" && continue am__quote=`sed -n 's/^am__quote = //p' < "$mf"` # Find all dependency output files, they are included files with # $(DEPDIR) in their names. We invoke sed twice because it is the # simplest approach to changing $(DEPDIR) to its actual value in the # expansion. for file in `sed -n " s/^$am__include $am__quote\(.*(DEPDIR).*\)$am__quote"'$/\1/p' <"$mf" | \ sed -e 's/\$(DEPDIR)/'"$DEPDIR"'/g'`; do # Make sure the directory exists. test -f "$dirpart/$file" && continue fdir=`$as_dirname -- "$file" || $as_expr X"$file" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ X"$file" : 'X\(//\)[^/]' \| \ X"$file" : 'X\(//\)$' \| \ X"$file" : 'X\(/\)' \| . 2>/dev/null || $as_echo X"$file" | sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ s//\1/ q } /^X\(\/\/\)[^/].*/{ s//\1/ q } /^X\(\/\/\)$/{ s//\1/ q } /^X\(\/\).*/{ s//\1/ q } s/.*/./; q'` as_dir=$dirpart/$fdir; as_fn_mkdir_p # echo "creating $dirpart/$file" echo '# dummy' > "$dirpart/$file" done done } ;; "libtool":C) # See if we are running on zsh, and set the options that allow our # commands through without removal of \ escapes. if test -n "${ZSH_VERSION+set}"; then setopt NO_GLOB_SUBST fi cfgfile=${ofile}T trap "$RM \"$cfgfile\"; exit 1" 1 2 15 $RM "$cfgfile" cat <<_LT_EOF >> "$cfgfile" #! $SHELL # Generated automatically by $as_me ($PACKAGE) $VERSION # Libtool was configured on host `(hostname || uname -n) 2>/dev/null | sed 1q`: # NOTE: Changes made to this file will be lost: look at ltmain.sh. # Provide generalized library-building support services. # Written by Gordon Matzigkeit, 1996 # Copyright (C) 2014 Free Software Foundation, Inc. # This is free software; see the source for copying conditions. There is NO # warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # GNU Libtool is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of of the License, or # (at your option) any later version. # # As a special exception to the GNU General Public License, if you # distribute this file as part of a program or library that is built # using GNU Libtool, you may include this file under the same # distribution terms that you use for the rest of that program. # # GNU Libtool is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # The names of the tagged configurations supported by this script. available_tags='CXX ' # Configured defaults for sys_lib_dlsearch_path munging. : \${LT_SYS_LIBRARY_PATH="$configure_time_lt_sys_library_path"} # ### BEGIN LIBTOOL CONFIG # Which release of libtool.m4 was used? macro_version=$macro_version macro_revision=$macro_revision # Whether or not to build shared libraries. build_libtool_libs=$enable_shared # Whether or not to build static libraries. build_old_libs=$enable_static # What type of objects to build. pic_mode=$pic_mode # Whether or not to optimize for fast installation. fast_install=$enable_fast_install # Shared archive member basename,for filename based shared library versioning on AIX. shared_archive_member_spec=$shared_archive_member_spec # Shell to use when invoking shell scripts. SHELL=$lt_SHELL # An echo program that protects backslashes. ECHO=$lt_ECHO # The PATH separator for the build system. PATH_SEPARATOR=$lt_PATH_SEPARATOR # The host system. host_alias=$host_alias host=$host host_os=$host_os # The build system. build_alias=$build_alias build=$build build_os=$build_os # A sed program that does not truncate output. SED=$lt_SED # Sed that helps us avoid accidentally triggering echo(1) options like -n. Xsed="\$SED -e 1s/^X//" # A grep program that handles long lines. GREP=$lt_GREP # An ERE matcher. EGREP=$lt_EGREP # A literal string matcher. FGREP=$lt_FGREP # A BSD- or MS-compatible name lister. NM=$lt_NM # Whether we need soft or hard links. LN_S=$lt_LN_S # What is the maximum length of a command? max_cmd_len=$max_cmd_len # Object file suffix (normally "o"). objext=$ac_objext # Executable file suffix (normally ""). exeext=$exeext # whether the shell understands "unset". lt_unset=$lt_unset # turn spaces into newlines. SP2NL=$lt_lt_SP2NL # turn newlines into spaces. NL2SP=$lt_lt_NL2SP # convert \$build file names to \$host format. to_host_file_cmd=$lt_cv_to_host_file_cmd # convert \$build files to toolchain format. to_tool_file_cmd=$lt_cv_to_tool_file_cmd # An object symbol dumper. OBJDUMP=$lt_OBJDUMP # Method to check whether dependent libraries are shared objects. deplibs_check_method=$lt_deplibs_check_method # Command to use when deplibs_check_method = "file_magic". file_magic_cmd=$lt_file_magic_cmd # How to find potential files when deplibs_check_method = "file_magic". file_magic_glob=$lt_file_magic_glob # Find potential files using nocaseglob when deplibs_check_method = "file_magic". want_nocaseglob=$lt_want_nocaseglob # DLL creation program. DLLTOOL=$lt_DLLTOOL # Command to associate shared and link libraries. sharedlib_from_linklib_cmd=$lt_sharedlib_from_linklib_cmd # The archiver. AR=$lt_AR # Flags to create an archive. AR_FLAGS=$lt_AR_FLAGS # How to feed a file listing to the archiver. archiver_list_spec=$lt_archiver_list_spec # A symbol stripping program. STRIP=$lt_STRIP # Commands used to install an old-style archive. RANLIB=$lt_RANLIB old_postinstall_cmds=$lt_old_postinstall_cmds old_postuninstall_cmds=$lt_old_postuninstall_cmds # Whether to use a lock for old archive extraction. lock_old_archive_extraction=$lock_old_archive_extraction # A C compiler. LTCC=$lt_CC # LTCC compiler flags. LTCFLAGS=$lt_CFLAGS # Take the output of nm and produce a listing of raw symbols and C names. global_symbol_pipe=$lt_lt_cv_sys_global_symbol_pipe # Transform the output of nm in a proper C declaration. global_symbol_to_cdecl=$lt_lt_cv_sys_global_symbol_to_cdecl # Transform the output of nm into a list of symbols to manually relocate. global_symbol_to_import=$lt_lt_cv_sys_global_symbol_to_import # Transform the output of nm in a C name address pair. global_symbol_to_c_name_address=$lt_lt_cv_sys_global_symbol_to_c_name_address # Transform the output of nm in a C name address pair when lib prefix is needed. global_symbol_to_c_name_address_lib_prefix=$lt_lt_cv_sys_global_symbol_to_c_name_address_lib_prefix # The name lister interface. nm_interface=$lt_lt_cv_nm_interface # Specify filename containing input files for \$NM. nm_file_list_spec=$lt_nm_file_list_spec # The root where to search for dependent libraries,and where our libraries should be installed. lt_sysroot=$lt_sysroot # Command to truncate a binary pipe. lt_truncate_bin=$lt_lt_cv_truncate_bin # The name of the directory that contains temporary libtool files. objdir=$objdir # Used to examine libraries when file_magic_cmd begins with "file". MAGIC_CMD=$MAGIC_CMD # Must we lock files when doing compilation? need_locks=$lt_need_locks # Manifest tool. MANIFEST_TOOL=$lt_MANIFEST_TOOL # Tool to manipulate archived DWARF debug symbol files on Mac OS X. DSYMUTIL=$lt_DSYMUTIL # Tool to change global to local symbols on Mac OS X. NMEDIT=$lt_NMEDIT # Tool to manipulate fat objects and archives on Mac OS X. LIPO=$lt_LIPO # ldd/readelf like tool for Mach-O binaries on Mac OS X. OTOOL=$lt_OTOOL # ldd/readelf like tool for 64 bit Mach-O binaries on Mac OS X 10.4. OTOOL64=$lt_OTOOL64 # Old archive suffix (normally "a"). libext=$libext # Shared library suffix (normally ".so"). shrext_cmds=$lt_shrext_cmds # The commands to extract the exported symbol list from a shared archive. extract_expsyms_cmds=$lt_extract_expsyms_cmds # Variables whose values should be saved in libtool wrapper scripts and # restored at link time. variables_saved_for_relink=$lt_variables_saved_for_relink # Do we need the "lib" prefix for modules? need_lib_prefix=$need_lib_prefix # Do we need a version for libraries? need_version=$need_version # Library versioning type. version_type=$version_type # Shared library runtime path variable. runpath_var=$runpath_var # Shared library path variable. shlibpath_var=$shlibpath_var # Is shlibpath searched before the hard-coded library search path? shlibpath_overrides_runpath=$shlibpath_overrides_runpath # Format of library name prefix. libname_spec=$lt_libname_spec # List of archive names. First name is the real one, the rest are links. # The last name is the one that the linker finds with -lNAME library_names_spec=$lt_library_names_spec # The coded name of the library, if different from the real name. soname_spec=$lt_soname_spec # Permission mode override for installation of shared libraries. install_override_mode=$lt_install_override_mode # Command to use after installation of a shared archive. postinstall_cmds=$lt_postinstall_cmds # Command to use after uninstallation of a shared archive. postuninstall_cmds=$lt_postuninstall_cmds # Commands used to finish a libtool library installation in a directory. finish_cmds=$lt_finish_cmds # As "finish_cmds", except a single script fragment to be evaled but # not shown. finish_eval=$lt_finish_eval # Whether we should hardcode library paths into libraries. hardcode_into_libs=$hardcode_into_libs # Compile-time system search path for libraries. sys_lib_search_path_spec=$lt_sys_lib_search_path_spec # Detected run-time system search path for libraries. sys_lib_dlsearch_path_spec=$lt_configure_time_dlsearch_path # Explicit LT_SYS_LIBRARY_PATH set during ./configure time. configure_time_lt_sys_library_path=$lt_configure_time_lt_sys_library_path # Whether dlopen is supported. dlopen_support=$enable_dlopen # Whether dlopen of programs is supported. dlopen_self=$enable_dlopen_self # Whether dlopen of statically linked programs is supported. dlopen_self_static=$enable_dlopen_self_static # Commands to strip libraries. old_striplib=$lt_old_striplib striplib=$lt_striplib # The linker used to build libraries. LD=$lt_LD # How to create reloadable object files. reload_flag=$lt_reload_flag reload_cmds=$lt_reload_cmds # Commands used to build an old-style archive. old_archive_cmds=$lt_old_archive_cmds # A language specific compiler. CC=$lt_compiler # Is the compiler the GNU compiler? with_gcc=$GCC # Compiler flag to turn off builtin functions. no_builtin_flag=$lt_lt_prog_compiler_no_builtin_flag # Additional compiler flags for building library objects. pic_flag=$lt_lt_prog_compiler_pic # How to pass a linker flag through the compiler. wl=$lt_lt_prog_compiler_wl # Compiler flag to prevent dynamic linking. link_static_flag=$lt_lt_prog_compiler_static # Does compiler simultaneously support -c and -o options? compiler_c_o=$lt_lt_cv_prog_compiler_c_o # Whether or not to add -lc for building shared libraries. build_libtool_need_lc=$archive_cmds_need_lc # Whether or not to disallow shared libs when runtime libs are static. allow_libtool_libs_with_static_runtimes=$enable_shared_with_static_runtimes # Compiler flag to allow reflexive dlopens. export_dynamic_flag_spec=$lt_export_dynamic_flag_spec # Compiler flag to generate shared objects directly from archives. whole_archive_flag_spec=$lt_whole_archive_flag_spec # Whether the compiler copes with passing no objects directly. compiler_needs_object=$lt_compiler_needs_object # Create an old-style archive from a shared archive. old_archive_from_new_cmds=$lt_old_archive_from_new_cmds # Create a temporary old-style archive to link instead of a shared archive. old_archive_from_expsyms_cmds=$lt_old_archive_from_expsyms_cmds # Commands used to build a shared archive. archive_cmds=$lt_archive_cmds archive_expsym_cmds=$lt_archive_expsym_cmds # Commands used to build a loadable module if different from building # a shared archive. module_cmds=$lt_module_cmds module_expsym_cmds=$lt_module_expsym_cmds # Whether we are building with GNU ld or not. with_gnu_ld=$lt_with_gnu_ld # Flag that allows shared libraries with undefined symbols to be built. allow_undefined_flag=$lt_allow_undefined_flag # Flag that enforces no undefined symbols. no_undefined_flag=$lt_no_undefined_flag # Flag to hardcode \$libdir into a binary during linking. # This must work even if \$libdir does not exist hardcode_libdir_flag_spec=$lt_hardcode_libdir_flag_spec # Whether we need a single "-rpath" flag with a separated argument. hardcode_libdir_separator=$lt_hardcode_libdir_separator # Set to "yes" if using DIR/libNAME\$shared_ext during linking hardcodes # DIR into the resulting binary. hardcode_direct=$hardcode_direct # Set to "yes" if using DIR/libNAME\$shared_ext during linking hardcodes # DIR into the resulting binary and the resulting library dependency is # "absolute",i.e impossible to change by setting \$shlibpath_var if the # library is relocated. hardcode_direct_absolute=$hardcode_direct_absolute # Set to "yes" if using the -LDIR flag during linking hardcodes DIR # into the resulting binary. hardcode_minus_L=$hardcode_minus_L # Set to "yes" if using SHLIBPATH_VAR=DIR during linking hardcodes DIR # into the resulting binary. hardcode_shlibpath_var=$hardcode_shlibpath_var # Set to "yes" if building a shared library automatically hardcodes DIR # into the library and all subsequent libraries and executables linked # against it. hardcode_automatic=$hardcode_automatic # Set to yes if linker adds runtime paths of dependent libraries # to runtime path list. inherit_rpath=$inherit_rpath # Whether libtool must link a program against all its dependency libraries. link_all_deplibs=$link_all_deplibs # Set to "yes" if exported symbols are required. always_export_symbols=$always_export_symbols # The commands to list exported symbols. export_symbols_cmds=$lt_export_symbols_cmds # Symbols that should not be listed in the preloaded symbols. exclude_expsyms=$lt_exclude_expsyms # Symbols that must always be exported. include_expsyms=$lt_include_expsyms # Commands necessary for linking programs (against libraries) with templates. prelink_cmds=$lt_prelink_cmds # Commands necessary for finishing linking programs. postlink_cmds=$lt_postlink_cmds # Specify filename containing input files. file_list_spec=$lt_file_list_spec # How to hardcode a shared library path into an executable. hardcode_action=$hardcode_action # The directories searched by this compiler when creating a shared library. compiler_lib_search_dirs=$lt_compiler_lib_search_dirs # Dependencies to place before and after the objects being linked to # create a shared library. predep_objects=$lt_predep_objects postdep_objects=$lt_postdep_objects predeps=$lt_predeps postdeps=$lt_postdeps # The library search path used internally by the compiler when linking # a shared library. compiler_lib_search_path=$lt_compiler_lib_search_path # ### END LIBTOOL CONFIG _LT_EOF cat <<'_LT_EOF' >> "$cfgfile" # ### BEGIN FUNCTIONS SHARED WITH CONFIGURE # func_munge_path_list VARIABLE PATH # ----------------------------------- # VARIABLE is name of variable containing _space_ separated list of # directories to be munged by the contents of PATH, which is string # having a format: # "DIR[:DIR]:" # string "DIR[ DIR]" will be prepended to VARIABLE # ":DIR[:DIR]" # string "DIR[ DIR]" will be appended to VARIABLE # "DIRP[:DIRP]::[DIRA:]DIRA" # string "DIRP[ DIRP]" will be prepended to VARIABLE and string # "DIRA[ DIRA]" will be appended to VARIABLE # "DIR[:DIR]" # VARIABLE will be replaced by "DIR[ DIR]" func_munge_path_list () { case x$2 in x) ;; *:) eval $1=\"`$ECHO $2 | $SED 's/:/ /g'` \$$1\" ;; x:*) eval $1=\"\$$1 `$ECHO $2 | $SED 's/:/ /g'`\" ;; *::*) eval $1=\"\$$1\ `$ECHO $2 | $SED -e 's/.*:://' -e 's/:/ /g'`\" eval $1=\"`$ECHO $2 | $SED -e 's/::.*//' -e 's/:/ /g'`\ \$$1\" ;; *) eval $1=\"`$ECHO $2 | $SED 's/:/ /g'`\" ;; esac } # Calculate cc_basename. Skip known compiler wrappers and cross-prefix. func_cc_basename () { for cc_temp in $*""; do case $cc_temp in compile | *[\\/]compile | ccache | *[\\/]ccache ) ;; distcc | *[\\/]distcc | purify | *[\\/]purify ) ;; \-*) ;; *) break;; esac done func_cc_basename_result=`$ECHO "$cc_temp" | $SED "s%.*/%%; s%^$host_alias-%%"` } # ### END FUNCTIONS SHARED WITH CONFIGURE _LT_EOF case $host_os in aix3*) cat <<\_LT_EOF >> "$cfgfile" # AIX sometimes has problems with the GCC collect2 program. For some # reason, if we set the COLLECT_NAMES environment variable, the problems # vanish in a puff of smoke. if test set != "${COLLECT_NAMES+set}"; then COLLECT_NAMES= export COLLECT_NAMES fi _LT_EOF ;; esac ltmain=$ac_aux_dir/ltmain.sh # We use sed instead of cat because bash on DJGPP gets confused if # if finds mixed CR/LF and LF-only lines. Since sed operates in # text mode, it properly converts lines to CR/LF. This bash problem # is reportedly fixed, but why not run on old versions too? sed '$q' "$ltmain" >> "$cfgfile" \ || (rm -f "$cfgfile"; exit 1) mv -f "$cfgfile" "$ofile" || (rm -f "$ofile" && cp "$cfgfile" "$ofile" && rm -f "$cfgfile") chmod +x "$ofile" cat <<_LT_EOF >> "$ofile" # ### BEGIN LIBTOOL TAG CONFIG: CXX # The linker used to build libraries. LD=$lt_LD_CXX # How to create reloadable object files. reload_flag=$lt_reload_flag_CXX reload_cmds=$lt_reload_cmds_CXX # Commands used to build an old-style archive. old_archive_cmds=$lt_old_archive_cmds_CXX # A language specific compiler. CC=$lt_compiler_CXX # Is the compiler the GNU compiler? with_gcc=$GCC_CXX # Compiler flag to turn off builtin functions. no_builtin_flag=$lt_lt_prog_compiler_no_builtin_flag_CXX # Additional compiler flags for building library objects. pic_flag=$lt_lt_prog_compiler_pic_CXX # How to pass a linker flag through the compiler. wl=$lt_lt_prog_compiler_wl_CXX # Compiler flag to prevent dynamic linking. link_static_flag=$lt_lt_prog_compiler_static_CXX # Does compiler simultaneously support -c and -o options? compiler_c_o=$lt_lt_cv_prog_compiler_c_o_CXX # Whether or not to add -lc for building shared libraries. build_libtool_need_lc=$archive_cmds_need_lc_CXX # Whether or not to disallow shared libs when runtime libs are static. allow_libtool_libs_with_static_runtimes=$enable_shared_with_static_runtimes_CXX # Compiler flag to allow reflexive dlopens. export_dynamic_flag_spec=$lt_export_dynamic_flag_spec_CXX # Compiler flag to generate shared objects directly from archives. whole_archive_flag_spec=$lt_whole_archive_flag_spec_CXX # Whether the compiler copes with passing no objects directly. compiler_needs_object=$lt_compiler_needs_object_CXX # Create an old-style archive from a shared archive. old_archive_from_new_cmds=$lt_old_archive_from_new_cmds_CXX # Create a temporary old-style archive to link instead of a shared archive. old_archive_from_expsyms_cmds=$lt_old_archive_from_expsyms_cmds_CXX # Commands used to build a shared archive. archive_cmds=$lt_archive_cmds_CXX archive_expsym_cmds=$lt_archive_expsym_cmds_CXX # Commands used to build a loadable module if different from building # a shared archive. module_cmds=$lt_module_cmds_CXX module_expsym_cmds=$lt_module_expsym_cmds_CXX # Whether we are building with GNU ld or not. with_gnu_ld=$lt_with_gnu_ld_CXX # Flag that allows shared libraries with undefined symbols to be built. allow_undefined_flag=$lt_allow_undefined_flag_CXX # Flag that enforces no undefined symbols. no_undefined_flag=$lt_no_undefined_flag_CXX # Flag to hardcode \$libdir into a binary during linking. # This must work even if \$libdir does not exist hardcode_libdir_flag_spec=$lt_hardcode_libdir_flag_spec_CXX # Whether we need a single "-rpath" flag with a separated argument. hardcode_libdir_separator=$lt_hardcode_libdir_separator_CXX # Set to "yes" if using DIR/libNAME\$shared_ext during linking hardcodes # DIR into the resulting binary. hardcode_direct=$hardcode_direct_CXX # Set to "yes" if using DIR/libNAME\$shared_ext during linking hardcodes # DIR into the resulting binary and the resulting library dependency is # "absolute",i.e impossible to change by setting \$shlibpath_var if the # library is relocated. hardcode_direct_absolute=$hardcode_direct_absolute_CXX # Set to "yes" if using the -LDIR flag during linking hardcodes DIR # into the resulting binary. hardcode_minus_L=$hardcode_minus_L_CXX # Set to "yes" if using SHLIBPATH_VAR=DIR during linking hardcodes DIR # into the resulting binary. hardcode_shlibpath_var=$hardcode_shlibpath_var_CXX # Set to "yes" if building a shared library automatically hardcodes DIR # into the library and all subsequent libraries and executables linked # against it. hardcode_automatic=$hardcode_automatic_CXX # Set to yes if linker adds runtime paths of dependent libraries # to runtime path list. inherit_rpath=$inherit_rpath_CXX # Whether libtool must link a program against all its dependency libraries. link_all_deplibs=$link_all_deplibs_CXX # Set to "yes" if exported symbols are required. always_export_symbols=$always_export_symbols_CXX # The commands to list exported symbols. export_symbols_cmds=$lt_export_symbols_cmds_CXX # Symbols that should not be listed in the preloaded symbols. exclude_expsyms=$lt_exclude_expsyms_CXX # Symbols that must always be exported. include_expsyms=$lt_include_expsyms_CXX # Commands necessary for linking programs (against libraries) with templates. prelink_cmds=$lt_prelink_cmds_CXX # Commands necessary for finishing linking programs. postlink_cmds=$lt_postlink_cmds_CXX # Specify filename containing input files. file_list_spec=$lt_file_list_spec_CXX # How to hardcode a shared library path into an executable. hardcode_action=$hardcode_action_CXX # The directories searched by this compiler when creating a shared library. compiler_lib_search_dirs=$lt_compiler_lib_search_dirs_CXX # Dependencies to place before and after the objects being linked to # create a shared library. predep_objects=$lt_predep_objects_CXX postdep_objects=$lt_postdep_objects_CXX predeps=$lt_predeps_CXX postdeps=$lt_postdeps_CXX # The library search path used internally by the compiler when linking # a shared library. compiler_lib_search_path=$lt_compiler_lib_search_path_CXX # ### END LIBTOOL TAG CONFIG: CXX _LT_EOF ;; esac done # for ac_tag as_fn_exit 0 _ACEOF ac_clean_files=$ac_clean_files_save test $ac_write_fail = 0 || as_fn_error $? "write failure creating $CONFIG_STATUS" "$LINENO" 5 # configure is writing to config.log, and then calls config.status. # config.status does its own redirection, appending to config.log. # Unfortunately, on DOS this fails, as config.log is still kept open # by configure, so config.status won't be able to write to it; its # output is simply discarded. So we exec the FD to /dev/null, # effectively closing config.log, so it can be properly (re)opened and # appended to by config.status. When coming back to configure, we # need to make the FD available again. if test "$no_create" != yes; then ac_cs_success=: ac_config_status_args= test "$silent" = yes && ac_config_status_args="$ac_config_status_args --quiet" exec 5>/dev/null $SHELL $CONFIG_STATUS $ac_config_status_args || ac_cs_success=false exec 5>>config.log # Use ||, not &&, to avoid exiting from the if with $? = 1, which # would make configure fail if this is the last instruction. $ac_cs_success || as_fn_exit 1 fi if test -n "$ac_unrecognized_opts" && test "$enable_option_checking" != no; then { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: unrecognized options: $ac_unrecognized_opts" >&5 $as_echo "$as_me: WARNING: unrecognized options: $ac_unrecognized_opts" >&2;} fi vowpal-wabbit-8.5.0.dfsg1/configure.ac000066400000000000000000000042551323713302100175740ustar00rootroot00000000000000AC_INIT([vowpal_wabbit], [8.5.0], [jl@hunch.net], [vowpal_wabbit], [https://github.com/JohnLangford/vowpal_wabbit]) AC_CONFIG_HEADERS(vowpalwabbit/config.h) AM_INIT_AUTOMAKE([subdir-objects foreign]) AC_CONFIG_MACRO_DIR([acinclude.d]) # if CXXFLAGS is not given by user, don't use default of "-O2 -g" : ${CXXFLAGS=""} AC_PROG_CXX AC_PROG_CXXCPP AC_LANG_CPLUSPLUS AC_CANONICAL_HOST AC_PROG_INSTALL AM_PROG_LIBTOOL AX_CXX_COMPILE_STDCXX_11 AC_HEADER_STDC AX_BOOST_BASE([1.0], [], [ AC_MSG_ERROR([Could not find a valid version of boost.]) ]) AX_BOOST_PROGRAM_OPTIONS() AX_CHECK_ZLIB() ZLIB_CPPFLAGS="-I${ZLIB_HOME}/include" ZLIB_LDFLAGS="-L${ZLIB_HOME}/lib -lz" AC_SUBST(ZLIB_CPPFLAGS) AC_SUBST(ZLIB_LDFLAGS) PTHREAD_LIBS=-lpthread AX_PTHREAD([], [ AC_MSG_ERROR([Could not find posix thread library.]) ]) nitpick=false AC_ARG_ENABLE([nitpicking], AC_HELP_STRING([--enable-nitpicking],[make compiler warn about possible problems]), [ test "$enableval" = "no" || nitpick=true ] ) AM_CONDITIONAL(NITPICK, test x$nitpick = xtrue) parallelize=false AC_ARG_ENABLE([parallelization],AC_HELP_STRING([--enable-parallelization],[enable openmp]),[ test "$enableval" = "no" || parallelize=true ]) AM_CONDITIONAL(PARALLELIZE, test x$parallelize = xtrue) profile=false AC_ARG_ENABLE([profiling], AC_HELP_STRING([--enable-profiling], [add -pg to C++ compiler flags]), [ test "$enableval" = "no" || profile=true ]) AM_CONDITIONAL(PROFILE, test x$profile = xtrue) vwbug=false AC_ARG_ENABLE([debug], AC_HELP_STRING([--enable-debug], [Enable debugging, disable optimization in the compiler]), [ test "$enableval" = "no" || vwbug=true ]) AM_CONDITIONAL(VWBUG, test x$vwbug = xtrue) clang_libcxx=false AC_ARG_ENABLE([libc++], AC_HELP_STRING([--enable-libc++],[use clang's libc++ vs. g++ libstdc++ headers/libraries]), [ test "$enableval" = "no" || clang_libcxx=true]) AM_CONDITIONAL([CLANG_LIBCXX], test x$clang_libcxx = xtrue) case $host in *-*-freebsd*) freebsd=true ;; *) freebsd=false ;; esac AM_CONDITIONAL(FREEBSD, test x$freebsd = xtrue) AC_CONFIG_FILES([ Makefile vowpalwabbit/Makefile cluster/Makefile library/Makefile libvw.pc libvw_c_wrapper.pc ]) AC_OUTPUT vowpal-wabbit-8.5.0.dfsg1/cs/000077500000000000000000000000001323713302100157055ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/azure/000077500000000000000000000000001323713302100170335ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/azure/ApplicationInsights.config000066400000000000000000000002101323713302100241670ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/azure/Join/000077500000000000000000000000001323713302100177325ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/azure/Join/QueueDictionary.cs000066400000000000000000000024671323713302100234040ustar00rootroot00000000000000using System; using System.Collections.Generic; namespace VowpalWabbit.Azure.Join { public class QueueDictionary { public class Item { public TKey Key { get; set; } public TValue Value { get; set; } } private readonly LinkedList queue = new LinkedList(); private readonly Dictionary> index = new Dictionary>(); public IEnumerable DequeueIf(Predicate pred) { while (queue.First != null && pred(queue.First.Value.Value)) { var item = queue.First.Value; queue.RemoveFirst(); index.Remove(item.Key); yield return item; } } public TValue Remove(TKey key) { LinkedListNode node; if (!index.TryGetValue(key, out node)) return default(TValue); var value = node.Value.Value; queue.Remove(node); index.Remove(key); return value; } public void Enqueue(TKey key, TValue value) { var node = queue.AddLast(new Item { Key = key, Value = value }); index.Add(key, node); } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Properties/000077500000000000000000000000001323713302100211675ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/azure/Properties/AssemblyInfo.cs000066400000000000000000000022131323713302100241070ustar00rootroot00000000000000//------------------------------------------------------------------------------ // // This code was generated by a tool. // Runtime Version:4.0.30319.42000 // // Changes to this file may cause incorrect behavior and will be lost if // the code is regenerated. // //------------------------------------------------------------------------------ [assembly: System.Reflection.AssemblyTitle("Vowpal Wabbit")] [assembly: System.Reflection.AssemblyDescription("Vowpal Wabbit")] [assembly: System.Reflection.AssemblyCompany("Microsoft Corp")] [assembly: System.Reflection.AssemblyProduct("Vowpal Wabbit")] [assembly: System.Reflection.AssemblyCopyright("Copyright (C) Microsoft Corp 2012-2016, Yahoo! Inc. 2007-2012, and many individua" + "l contributors. All rights reserved")] [assembly: System.Runtime.InteropServices.ComVisible(false)] [assembly: System.CLSCompliant(false)] [assembly: System.Runtime.InteropServices.Guid("6a577997-af00-4ca0-8453-fdc8bbdf2a57")] [assembly: System.Reflection.AssemblyVersion("8.3.0.9")] [assembly: System.Reflection.AssemblyFileVersion("8.3.0.9")] vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/000077500000000000000000000000001323713302100204375ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/Checkpoint/000077500000000000000000000000001323713302100225265ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/Checkpoint/CountingCheckpointPolicy.cs000066400000000000000000000037241323713302100300410ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- namespace VW.Azure.Trainer.Checkpoint { /// /// Implements an example count based checkpoint policy. /// public class CountingCheckpointPolicy : ICheckpointPolicy { private readonly int exampleSyncCount; private int exampleCount; /// /// Initializes a new instance. /// public CountingCheckpointPolicy(int exampleSyncCount) { this.exampleSyncCount = exampleSyncCount; } /// /// Return true if the trainer should checkpoint the model, false otherwise. /// /// Number of examples since last checkpoint. public bool ShouldCheckpointAfterExample(int examples) { this.exampleCount += examples; if (this.exampleCount >= this.exampleSyncCount) { this.exampleCount %= this.exampleSyncCount; return true; } return false; } /// /// Reset checkpoint policy state. /// public void Reset() { this.exampleCount = 0; } /// /// Serialize to string for logging. /// public override string ToString() { return $"CountingCheckpointPolicy: {this.exampleSyncCount}"; } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/Checkpoint/ICheckpointPolicy.cs000066400000000000000000000020421323713302100264330ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- namespace VW.Azure.Trainer.Checkpoint { /// /// Interface for model checkpoint policies. /// public interface ICheckpointPolicy { /// /// Return true if the trainer should checkpoint the model, false otherwise. /// /// Number of examples since last checkpoint. bool ShouldCheckpointAfterExample(int examples); /// /// Reset checkpoint policy state. /// void Reset(); } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/Checkpoint/TimespanCheckpointPolicy.cs000066400000000000000000000041751323713302100300340ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Diagnostics; namespace VW.Azure.Trainer.Checkpoint { /// /// Implements a wallclock time based checkpoint policy. /// public class IntervalCheckpointPolicy : ICheckpointPolicy { private Stopwatch stopwatch; private TimeSpan checkpointInterval; /// /// Initializes a new instance. /// public IntervalCheckpointPolicy(TimeSpan checkpointInterval) { this.stopwatch = Stopwatch.StartNew(); this.checkpointInterval = checkpointInterval; } /// /// Return true if the trainer should checkpoint the model, false otherwise. /// /// Number of examples since last checkpoint. public bool ShouldCheckpointAfterExample(int examples) { // call checkpoint every 5 minutes, so that worker can resume processing from 5 minutes back if it restarts. if (this.stopwatch.Elapsed > checkpointInterval) { this.stopwatch.Restart(); return true; } return false; } /// /// Reset checkpoint policy state. /// public void Reset() { this.stopwatch.Restart(); } /// /// Serialize to string for logging. /// public override string ToString() { return $"IntervalCheckpointPolicy: {this.checkpointInterval}"; } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/Data/000077500000000000000000000000001323713302100213105ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/Data/CheckpointData.cs000066400000000000000000000017461323713302100245300ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; namespace VW.Azure.Trainer.Data { internal sealed class CheckpointData { internal byte[] Model { get; set; } internal byte[] EvalModel { get; set; } internal int TrackbackCount { get; set; } internal string TrackbackList { get; set; } internal bool UpdateClientModel { get; set; } internal string State { get; set; } internal string Timestamp { get; set; } internal DateTime StartDateTime { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/Data/CheckpointEvaluateTriggerEvent.cs000066400000000000000000000013261323713302100277450ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- namespace VW.Azure.Trainer.Data { internal sealed class CheckpointEvaluateTriggerEvent { public override string ToString() { return "Checkpoint evaluate request"; } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/Data/CheckpointTriggerEvent.cs000066400000000000000000000013621323713302100262560ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- namespace VW.Azure.Trainer.Data { internal sealed class CheckpointTriggerEvent { public override string ToString() { return "Checkpoint request"; } public bool UpdateClientModel { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/Data/PipelineData.cs000066400000000000000000000024731323713302100242040ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using VW; namespace VW.Azure.Trainer.Data { internal sealed class PipelineData { internal string JSON { get; set; } internal string Offset { get; set; } internal string PartitionKey { get; set; } internal string PartitionId { get; set; } internal string EventId { get; set; } internal DateTime Timestamp { get; set; } internal int[] Actions { get; set; } internal float[] Probabilities { get; set; } internal float ProbabilityOfDrop { get; set; } /// /// 1-based action mapping to tag. /// internal Dictionary ActionsTags { get; set; } = new Dictionary(); public VowpalWabbitExampleCollection Example { get; set; } } }vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/Data/TrainerResult.cs000066400000000000000000000055561323713302100244550ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Linq; using VW; using VW.Labels; namespace VW.Azure.Trainer.Data { internal sealed class TrainerResult { internal TrainerResult(ActionScore[] progressivePrediction, int[] observedActions, float[] observedProbabilities) { if (progressivePrediction == null) throw new ArgumentNullException(nameof(progressivePrediction)); if (observedActions == null) throw new ArgumentNullException(nameof(observedActions)); if (observedProbabilities == null) throw new ArgumentNullException(nameof(observedProbabilities)); if (observedActions.Length != observedProbabilities.Length) throw new ArgumentException($"Actions (length: {observedActions.Length}) and probabilities (length: {observedProbabilities.Length}) must be of equal length"); this.ProgressiveRanking = progressivePrediction.Select(a => (int)a.Action).ToArray(); var probabilitiesOrderedByRanking = progressivePrediction.Select(a => a.Score).ToArray(); this.ProgressiveProbabilities = new float[probabilitiesOrderedByRanking.Length]; for (int i = 0; i < probabilitiesOrderedByRanking.Length; i++) this.ProgressiveProbabilities[ProgressiveRanking[i]] = probabilitiesOrderedByRanking[i]; // Ranking is 0-based this.ObservedRanking = observedActions; this.ObservedProbabilities = new float[observedProbabilities.Length]; for (int i = 0; i < observedActions.Length; i++) this.ObservedProbabilities[observedActions[i] - 1] = observedProbabilities[i]; } internal ContextualBanditLabel Label { get; set; } internal TimeSpan Latency { get; set; } internal string PartitionKey { get; set; } internal string PartitionId { get; set; } internal int[] ProgressiveRanking { get; private set; } internal float[] ProgressiveProbabilities { get; private set; } internal int[] ObservedRanking { get; private set; } internal float[] ObservedProbabilities { get; private set; } internal float ProbabilityOfDrop { get; set; } internal Dictionary ActionsTags { get; set; } internal string EventId { get; set; } internal DateTime Timestamp { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/LearnEventProcessor.cs000066400000000000000000000051151323713302100247330ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Microsoft.ApplicationInsights; using Microsoft.ApplicationInsights.DataContracts; using Microsoft.ServiceBus.Messaging; using System.Collections.Generic; using System.Threading.Tasks; namespace VW.Azure.Trainer { internal sealed class LearnEventProcessor : IEventProcessor { private readonly TrainEventProcessorFactory parent; private readonly TelemetryClient telemetry; private readonly PerformanceCounters perfCounters; internal LearnEventProcessor(TrainEventProcessorFactory parent, PerformanceCounters perfCounters) { this.telemetry = new TelemetryClient(); this.telemetry.Context.Component.Version = "TrainEventProcessor v" + GetType().Assembly.GetName().Version; this.parent = parent; this.perfCounters = perfCounters; } public Task OpenAsync(PartitionContext context) { this.telemetry.TrackTrace( $"OpenPartition Id {context.Lease.PartitionId}", SeverityLevel.Information, new Dictionary { { "PartitionId", context.Lease.PartitionId }, { "Offset", context.Lease.Offset } }); this.perfCounters.EventHub_Processors.Increment(); return Task.FromResult(true); } public async Task ProcessEventsAsync(PartitionContext context, IEnumerable messages) { await this.parent.Stage0_Split(context, messages); } public Task CloseAsync(PartitionContext context, CloseReason reason) { this.telemetry.TrackTrace( $"ClosePartition {context.Lease.PartitionId}: {reason}", SeverityLevel.Information, new Dictionary { { "PartitionId", context.Lease.PartitionId }, { "Reason", reason.ToString() } }); this.perfCounters.EventHub_Processors.Decrement(); return Task.FromResult(true); } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/LearnEventProcessorFactory.cs000066400000000000000000000345261323713302100262730ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Microsoft.ApplicationInsights; using Microsoft.ApplicationInsights.DataContracts; using Microsoft.ServiceBus.Messaging; using System; using System.Linq; using System.Collections.Generic; using System.Threading; using System.Threading.Tasks; using System.Threading.Tasks.Dataflow; using VW; using Newtonsoft.Json; using System.IO; using System.Diagnostics; using System.Reactive.Linq; using System.Reactive.Subjects; using VW.Labels; using System.Text; using VW.Azure.Trainer; using VW.Azure.Trainer.Operations; using VW.Serializer; using VW.Azure.Trainer.Data; namespace VW.Azure { internal sealed class TrainEventProcessorFactory : IEventProcessorFactory, IDisposable { private readonly TelemetryClient telemetry; private readonly PerformanceCounters performanceCounters; private readonly Learner trainer; private EvalOperation evalOperation; private LatencyOperation latencyOperation; private TransformManyBlock deserializeBlock; private TransformManyBlock learnBlock; private ActionBlock checkpointBlock; private IDisposable checkpointTrigger; internal TrainEventProcessorFactory(OnlineTrainerSettingsInternal settings, Learner trainer, PerformanceCounters performanceCounters) { if (settings == null) throw new ArgumentNullException(nameof(settings)); if (trainer == null) throw new ArgumentNullException(nameof(trainer)); if (performanceCounters == null) throw new ArgumentNullException(nameof(performanceCounters)); this.trainer = trainer; this.performanceCounters = performanceCounters; this.telemetry = new TelemetryClient(); this.telemetry.Context.Component.Version = GetType().Assembly.GetName().Version.ToString(); this.evalOperation = new EvalOperation(settings, performanceCounters); this.latencyOperation = new LatencyOperation(); this.deserializeBlock = new TransformManyBlock( (Func>)this.Stage1_Deserialize, new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = 4, // Math.Max(2, Environment.ProcessorCount - 1), BoundedCapacity = 1024 }); this.deserializeBlock.Completion.Trace(this.telemetry, "Stage 1 - Deserialization"); this.learnBlock = new TransformManyBlock( (Func>)this.Stage2_ProcessEvent, new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = 1, BoundedCapacity = 1024 }); this.learnBlock.Completion.Trace(this.telemetry, "Stage 2 - Learning"); // trigger checkpoint checking every second this.checkpointTrigger = Observable.Interval(TimeSpan.FromSeconds(1)) .Select(_ => new CheckpointEvaluateTriggerEvent()) .Subscribe(this.learnBlock.AsObserver()); this.checkpointBlock = new ActionBlock( this.trainer.Checkpoint, new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = 1, BoundedCapacity = 4 }); this.learnBlock.Completion.Trace(this.telemetry, "Stage 3 - CheckPointing"); // setup pipeline this.deserializeBlock.LinkTo( this.learnBlock, new DataflowLinkOptions { PropagateCompletion = true }); this.learnBlock.LinkTo( this.evalOperation.TargetBlock, new DataflowLinkOptions { PropagateCompletion = true }, obj => obj is TrainerResult); this.learnBlock.LinkTo( this.checkpointBlock, new DataflowLinkOptions { PropagateCompletion = true }, obj => obj is CheckpointData); // consume all unmatched this.learnBlock.LinkTo(DataflowBlock.NullTarget()); } internal void UpdatePerformanceCounters() { this.performanceCounters.Stage1_JSON_Queue.RawValue = this.deserializeBlock.InputCount; this.performanceCounters.Stage2_Learn_Queue.RawValue = this.learnBlock.InputCount; this.performanceCounters.Stage3_Checkpoint_Queue.RawValue = this.checkpointBlock.InputCount; } internal async Task Stage0_Split(PartitionContext context, IEnumerable messages) { foreach (EventData eventData in messages) { try { using (var eventStream = eventData.GetBodyStream()) { using (var sr = new StreamReader(eventStream, Encoding.UTF8)) { string line; while ((line = await sr.ReadLineAsync()) != null) { var data = new PipelineData { JSON = line, PartitionId = context.Lease.PartitionId, PartitionKey = eventData.PartitionKey, Offset = eventData.Offset }; // TODO: ArrayBuffer to avoid string allocation... // also just send char ref + offset + length if (!await this.deserializeBlock.SendAsync(data)) this.telemetry.TrackTrace("Failed to enqueue data"); } this.performanceCounters.Stage0_IncomingBytesPerSec.IncrementBy(eventStream.Position); this.performanceCounters.Stage0_Batches_Size.IncrementBy(eventStream.Position); this.performanceCounters.Stage0_Batches_SizeBase.Increment(); } } this.performanceCounters.Stage0_BatchesPerSec.Increment(); this.performanceCounters.Stage0_Batches_Total.Increment(); } catch (Exception ex) { this.telemetry.TrackException(ex); } } } private static bool TryExtractProperty(VowpalWabbitJsonParseState state, string property, string expectedProperty, JsonToken expectedToken, Action success) { if (property.Equals(expectedProperty, StringComparison.OrdinalIgnoreCase)) { if (!state.Reader.Read() && state.Reader.TokenType != expectedToken) throw new VowpalWabbitJsonException(state.Reader, $"Property '{expectedProperty}' must be of type '{expectedToken}'"); success(state.Reader); return true; } return false; } private static bool TryExtractArrayProperty(VowpalWabbitJsonParseState state, string property, string expectedProperty, Action success) { return TryExtractProperty( state, property, expectedProperty, JsonToken.StartArray, reader => { success(JsonSerializer.CreateDefault().Deserialize(reader)); if (state.Reader.TokenType != JsonToken.EndArray && !reader.Read()) throw new VowpalWabbitJsonException(state.Reader, $"Property {expectedProperty} must end with 'EndArray'"); }); } private IEnumerable Stage1_Deserialize(PipelineData data) { try { using (var jsonReader = new JsonTextReader(new StringReader(data.JSON))) { //jsonReader.FloatParser = Util.ReadDoubleString; // jsonReader.ArrayPool = pool; VowpalWabbitJsonSerializer vwJsonSerializer = null; try { vwJsonSerializer = new VowpalWabbitJsonSerializer(this.trainer.VowpalWabbit, this.trainer.ReferenceResolver); vwJsonSerializer.RegisterExtension((state, property) => { if (TryExtractProperty(state, property, "_eventid", JsonToken.String, reader => data.EventId = (string)reader.Value)) return true; else if (TryExtractProperty(state, property, "_timestamp", JsonToken.Date, reader => data.Timestamp = (DateTime)reader.Value)) return true; else if (TryExtractProperty(state, property, "_ProbabilityOfDrop", JsonToken.Float, reader => data.ProbabilityOfDrop = (float)(reader.Value ?? 0f))) return true; else if (TryExtractArrayProperty(state, property, "_p", arr => data.Probabilities = arr)) return true; else if (TryExtractArrayProperty(state, property, "_a", arr => data.Actions = arr)) return true; else if (TryExtractProperty(state, property, "_tag", JsonToken.String, reader => data.ActionsTags.Add(state.MultiIndex + 1, (string)reader.Value))) return true; return false; }); data.Example = vwJsonSerializer.ParseAndCreate(jsonReader); if (data.Probabilities == null) throw new ArgumentNullException("Missing probabilities (_p)"); if (data.Actions == null) throw new ArgumentNullException("Missing actions (_a)"); if (data.Example == null) { // unable to create example due to missing data // will be trigger later vwJsonSerializer.UserContext = data.Example; // make sure the serialize is not deallocated vwJsonSerializer = null; } } finally { if (vwJsonSerializer != null) vwJsonSerializer.Dispose(); } performanceCounters.Stage1_JSON_DeserializePerSec.Increment(); // delayed if (data.Example == null) { this.performanceCounters.Feature_Requests_Pending.Increment(); yield break; } } } catch (Exception ex) { this.telemetry.TrackException(ex, new Dictionary { { "JSON", data.JSON } }); this.performanceCounters.Stage2_Faulty_Examples_Total.Increment(); this.performanceCounters.Stage2_Faulty_ExamplesPerSec.Increment(); yield break; } yield return data; } private IEnumerable Stage2_ProcessEvent(object evt) { // single threaded loop var eventHubExample = evt as PipelineData; if (eventHubExample != null) { var result = this.trainer.Learn(eventHubExample); // report latency this.latencyOperation.Process(result); yield return result; if (this.trainer.ShouldCheckpoint(1)) yield return this.trainer.CreateCheckpointData(updateClientModel: true); } else if (evt is CheckpointTriggerEvent) yield return this.trainer.CreateCheckpointData(updateClientModel: ((CheckpointTriggerEvent)evt).UpdateClientModel); else if (evt is CheckpointEvaluateTriggerEvent) { if (this.trainer.ShouldCheckpoint(0)) yield return this.trainer.CreateCheckpointData(updateClientModel: true); } else this.telemetry.TrackTrace($"Unsupported stage 2 event '{evt}'", SeverityLevel.Warning); } public IEventProcessor CreateEventProcessor(PartitionContext context) { return new LearnEventProcessor(this, this.performanceCounters); } public ITargetBlock LearnBlock { get { return this.learnBlock; } } /// /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources. /// public void Dispose() { if (this.checkpointTrigger != null) { this.checkpointTrigger.Dispose(); this.checkpointTrigger = null; } if (this.learnBlock != null) { // complete beginning of the pipeline this.deserializeBlock.Complete(); // wait at the end of the pipeline this.checkpointBlock.Completion.Wait(TimeSpan.FromMinutes(1)); } if (this.evalOperation != null) { this.evalOperation.Dispose(); this.evalOperation = null; } if (this.latencyOperation != null) { this.latencyOperation.Dispose(); this.latencyOperation = null; } } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/LearnEventProcessorHost.cs000066400000000000000000000300351323713302100255700ustar00rootroot00000000000000using Microsoft.ApplicationInsights; using Microsoft.ApplicationInsights.DataContracts; using Microsoft.ServiceBus; using Microsoft.ServiceBus.Messaging; using System; using System.Collections.Generic; using System.Reactive.Linq; using System.Threading; using System.Threading.Tasks; using System.Threading.Tasks.Dataflow; using VW.Azure.Trainer.Data; using VW.Azure.Trainer.Operations; using VW.Serializer; namespace VW.Azure.Trainer { /// /// Azure online trainer. /// public sealed class LearnEventProcessorHost : IDisposable { private readonly TelemetryClient telemetry; private readonly object managementLock = new object(); private TrainEventProcessorFactory trainProcessorFactory; private EventProcessorHost eventProcessorHost; private Learner trainer; private PerformanceCounters perfCounters; private SafeTimer perfUpdater; private DateTime? eventHubStartDateTimeUtc; /// /// Initializes a new instance. /// public LearnEventProcessorHost() { this.telemetry = new TelemetryClient(); // by default read from the beginning of Event Hubs event stream. this.eventHubStartDateTimeUtc = null; } /// /// Performance countners populated by online trainer. /// public PerformanceCounters PerformanceCounters { get { return this.perfCounters; } } /// /// Timestamp when the trainer was last started. /// public DateTime LastStartDateTimeUtc { get; private set; } internal object InitialOffsetProvider(string partition) { string offset; if (this.trainer.State.Partitions.TryGetValue(partition, out offset)) return offset; // either DateTime.UtcNow on reset or null if start the first time return this.eventHubStartDateTimeUtc; } /// /// Starts the trainer with given parameters. /// public async Task StartAsync(OnlineTrainerSettingsInternal settings) { await this.SafeExecute(async () => await this.StartInternalAsync(settings)); } /// /// Stops the trainer. /// /// public async Task StopAsync() { await this.SafeExecute(this.StopInternalAsync); } /// /// Restarts the trainer. /// public async Task Restart(OnlineTrainerSettingsInternal settings) { await this.SafeExecute(async () => await this.RestartInternalAsync(settings)); } /// /// Resets the trainers. /// public async Task ResetModelAsync(OnlineTrainerState state = null, byte[] model = null) { await this.SafeExecute(async () => await this.ResetInternalAsync(state, model)); } /// /// Forces model checkpointing. /// public async Task CheckpointAsync() { await this.SafeExecute(async () => await this.trainProcessorFactory.LearnBlock.SendAsync(new CheckpointTriggerEvent())); } private Task SafeExecute(Func action) { try { // need to do a lock as child tasks are interleaving lock (this.managementLock) { action().Wait(TimeSpan.FromMinutes(3)); } } catch (AggregateException ex) { foreach (var innerEx in ex.Flatten().InnerExceptions) this.telemetry.TrackException(innerEx); throw ex; } catch (Exception ex) { this.telemetry.TrackException(ex); throw ex; } return Task.FromResult(true); } private async Task ResetInternalAsync(OnlineTrainerState state = null, byte[] model = null) { if (this.trainer == null) { this.telemetry.TrackTrace("Online Trainer resetting skipped as trainer hasn't started yet.", SeverityLevel.Information); return; } var msg = "Online Trainer resetting"; bool updateClientModel = false; if (state != null) { msg += "; state supplied"; updateClientModel = true; } if (model != null) { msg += $"; model of size {model.Length} supplied."; updateClientModel = true; } this.telemetry.TrackTrace(msg, SeverityLevel.Information); var settings = this.trainer.Settings; await this.StopInternalAsync(); settings.ForceFreshStart = true; settings.CheckpointPolicy.Reset(); await this.StartInternalAsync(settings, state, model); // make sure we store this fresh model, in case we die we don't loose the reset await this.trainProcessorFactory.LearnBlock.SendAsync(new CheckpointTriggerEvent { UpdateClientModel = updateClientModel }); if (!updateClientModel) { // delete the currently deployed model, so the clients don't use the hold one var latestModel = await this.trainer.GetLatestModelBlob(); this.telemetry.TrackTrace($"Resetting client visible model: {latestModel.Uri}", SeverityLevel.Information); await latestModel.UploadFromByteArrayAsync(new byte[0], 0, 0); } } private async Task RestartInternalAsync(OnlineTrainerSettingsInternal settings) { this.telemetry.TrackTrace("Online Trainer restarting", SeverityLevel.Information); await this.StopInternalAsync(); await this.StartInternalAsync(settings); } private async Task StartInternalAsync(OnlineTrainerSettingsInternal settings, OnlineTrainerState state = null, byte[] model = null) { this.LastStartDateTimeUtc = DateTime.UtcNow; this.perfCounters = new PerformanceCounters(settings.Metadata.ApplicationID); // setup trainer this.trainer = new Learner(settings, this.DelayedExampleCallback, this.perfCounters); if (settings.ForceFreshStart || model != null) this.trainer.FreshStart(state, model); else await this.trainer.FindAndResumeFromState(); // setup factory this.trainProcessorFactory = new TrainEventProcessorFactory(settings, this.trainer, this.perfCounters); // setup host var serviceBusConnectionStringBuilder = new ServiceBusConnectionStringBuilder(settings.JoinedEventHubConnectionString); var joinedEventhubName = serviceBusConnectionStringBuilder.EntityPath; serviceBusConnectionStringBuilder.EntityPath = string.Empty; this.eventProcessorHost = new EventProcessorHost(settings.Metadata.ApplicationID, joinedEventhubName, settings.JoinedEventHubConsumerGroup, serviceBusConnectionStringBuilder.ToString(), settings.StorageConnectionString); // used by this.InitialOffsetProvider if no checkpointed state is found this.eventHubStartDateTimeUtc = settings.EventHubStartDateTimeUtc; await this.eventProcessorHost.RegisterEventProcessorFactoryAsync( this.trainProcessorFactory, new EventProcessorOptions { InitialOffsetProvider = this.InitialOffsetProvider }); // don't perform too often this.perfUpdater = new SafeTimer( TimeSpan.FromMilliseconds(500), this.UpdatePerformanceCounters); var vwArgs = this.trainer.VowpalWabbit.Arguments; this.telemetry.TrackTrace( "OnlineTrainer started", SeverityLevel.Information, new Dictionary { { "CheckpointPolicy", settings.CheckpointPolicy.ToString() }, { "VowpalWabbit", settings.Metadata.TrainArguments }, { "ExampleTracing", settings.EnableExampleTracing.ToString() }, { "LearningRate", vwArgs.LearningRate.ToString() }, { "PowerT", vwArgs.PowerT.ToString() } }); } private void UpdatePerformanceCounters() { lock (this.managementLock) { // make sure this is thread safe w.r.t reset/start/stop/... try { if (this.trainer != null && this.trainProcessorFactory != null) { this.trainer.UpdatePerformanceCounters(); this.trainProcessorFactory.UpdatePerformanceCounters(); } } catch (Exception ex) { this.telemetry.TrackException(ex); } } } private async Task StopInternalAsync() { this.telemetry.TrackTrace("OnlineTrainer stopping", SeverityLevel.Verbose); if (this.perfUpdater != null) { this.perfUpdater.Stop(TimeSpan.FromMinutes(1)); this.perfUpdater = null; } if (this.eventProcessorHost != null) { try { await this.eventProcessorHost.UnregisterEventProcessorAsync(); } catch (Exception ex) { this.telemetry.TrackException(ex); } this.eventProcessorHost = null; } if (this.trainProcessorFactory != null) { // flushes the pipeline this.trainProcessorFactory.Dispose(); this.trainProcessorFactory = null; } if (this.trainer != null) { this.trainer.Dispose(); this.trainer = null; } if (this.perfCounters != null) { this.perfCounters.Dispose(); this.perfCounters = null; } this.telemetry.TrackTrace("OnlineTrainer stopped", SeverityLevel.Verbose); } private void DelayedExampleCallback(VowpalWabbitJsonSerializer serializer) { try { this.perfCounters.Feature_Requests_Pending.IncrementBy(-1); var data = (PipelineData)serializer.UserContext; data.Example = serializer.CreateExamples(); // fire and forget // must not block to avoid dead lock this.trainProcessorFactory.LearnBlock .SendAsync(data) .ContinueWith(async ret => { if (!await ret) { this.telemetry.TrackTrace("Unable to enqueue delayed examples", SeverityLevel.Error); // since we couldn't enqueue, need to dispose here data.Example.Dispose(); } }); } catch (Exception e) { this.telemetry.TrackException(e); } finally { serializer.Dispose(); } } /// /// Dispose the trainer. /// public void Dispose() { try { this.StopAsync().Wait(TimeSpan.FromMinutes(1)); } catch (Exception ex) { this.telemetry.TrackException(ex); } } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/Learner.cs000066400000000000000000000074121323713302100223620ustar00rootroot00000000000000using Microsoft.ApplicationInsights; using Microsoft.WindowsAzure.Storage; using Microsoft.WindowsAzure.Storage.Blob; using System; using System.Collections.Generic; using System.Globalization; using System.Reactive.Linq; using System.Threading.Tasks; using VW; using VW.Serializer; namespace VW.Azure.Trainer { internal partial class Learner : IDisposable { internal const string StateBlobName = "state.json"; private readonly TelemetryClient telemetry; private CloudBlobClient blobClient; private VW.VowpalWabbit vw; private VowpalWabbitJsonReferenceResolver referenceResolver; private List trackbackList; private OnlineTrainerSettingsInternal settings; private OnlineTrainerState state; private readonly Action delayedExampleCallback; private DateTime startDateTime; private readonly PerformanceCounters perfCounters; //private VowpalWabbitThreadedLearning vwAllReduce; internal Learner(OnlineTrainerSettingsInternal settings, Action delayedExampleCallback, PerformanceCounters perfCounters) { this.telemetry = new TelemetryClient(); this.settings = settings; this.delayedExampleCallback = delayedExampleCallback; this.perfCounters = perfCounters; this.trackbackList = new List(); this.blobClient = CloudStorageAccount.Parse(settings.StorageConnectionString).CreateCloudBlobClient(); } internal void UpdatePerformanceCounters() { if (this.referenceResolver != null) { // don't do this too often as it grabs a lock var stats = this.referenceResolver.Statistics; this.perfCounters.Features_Cached.RawValue = stats.ItemCount; this.perfCounters.Feature_Requests_Pending.RawValue = stats.NumberOfOpenRequests; } } internal OnlineTrainerState State { get { return this.state; } } internal OnlineTrainerSettingsInternal Settings { get { return this.settings; } } /// /// create light-weight VW instance for example deserialization /// need thread-safe example pool wrapper as the examples are allocated and disposed on different threads /// internal VW.VowpalWabbit VowpalWabbit { get { return this.vw; } } internal VowpalWabbitJsonReferenceResolver ReferenceResolver { get { return this.referenceResolver; } } internal async Task GetLatestModelBlob() { var latestModelContainerName = OnlineTrainerSettings.ModelContainerName; var mwtModelContainer = this.blobClient.GetContainerReference(latestModelContainerName); await mwtModelContainer.CreateIfNotExistsAsync(); return mwtModelContainer.GetBlockBlobReference(OnlineTrainerSettings.LatestModelBlobName); } /// /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources. /// public void Dispose() { //if (this.vwAllReduce != null) //{ // this.vwAllReduce.Dispose(); // this.vwAllReduce = null; //} if (this.vw != null) { this.vw.Dispose(); this.vw = null; } if (this.referenceResolver != null) { this.referenceResolver.Dispose(); this.referenceResolver = null; } } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/OnlineTrainerSettings.cs000066400000000000000000000032321323713302100252600ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- namespace VW.Azure.Trainer { /// /// Settings for the online trainer. /// public class OnlineTrainerSettings { /// /// Azure storage container name containing the latest model. /// public const string ModelContainerName = "mwt-models"; /// /// Azure storage blob name of the latest model. /// public const string LatestModelBlobName = "current"; /// /// Azure storage container name containing the latest settings. /// public const string SettingsContainerName = "mwt-settings"; /// /// Azure storage blob name of the latest settings. /// public const string LatestClientSettingsBlobName = "client"; /// /// Application ID used by performance counter instance name. /// public string ApplicationID { get; set; } /// /// Training arguments to be used in training service. /// public string TrainArguments { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/OnlineTrainerSettingsInternal.cs000066400000000000000000000052721323713302100267630ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Microsoft.ServiceBus.Messaging; using System; using VW.Azure.Trainer.Checkpoint; namespace VW.Azure.Trainer { /// /// The internal trainer settings. /// public class OnlineTrainerSettingsInternal { /// /// The Azure storage container name used for model and state history. /// public string StorageContainerName { get; private set; } = "onlinetrainer"; /// /// External supplied meta data. /// public OnlineTrainerSettings Metadata { get; set; } /// /// The initial model the training run started from. /// public string InitialVowpalWabbitModel { get; set; } /// /// Azure storage connection string used for checkpointing. /// public string StorageConnectionString { get; set; } /// /// Input data Azure EventHub connection string. /// public string JoinedEventHubConnectionString { get; set; } /// /// Consumer group used for joined events. /// public string JoinedEventHubConsumerGroup { get; set; } = EventHubConsumerGroup.DefaultGroupName; /// /// Evaluation output Azure Eventhub connection string. /// public string EvalEventHubConnectionString { get; set; } /// /// Checkpoint policy. /// public ICheckpointPolicy CheckpointPolicy { get; set; } /// /// True if examples should be traced. /// public bool EnableExampleTracing { get; set; } /// /// Null will let the trainer read events earliest available timestamps in event hub input; /// Any other valid DateTime will let the trainer read events from that point in time. /// public DateTime? EventHubStartDateTimeUtc { get; set; } /// /// True if a fresh start was forced. /// internal bool ForceFreshStart { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/OnlineTrainerState.cs000066400000000000000000000042231323713302100245410ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Newtonsoft.Json; using System; using System.Collections.Generic; using System.Linq; namespace VW.Azure.Trainer { /// /// Online trainer state structure used to serialize state to state.json. /// public class OnlineTrainerState { /// /// Initializes a new instance. /// public OnlineTrainerState() { this.Partitions = new Dictionary(); this.PartitionsDateTime = new Dictionary(); } /// /// Current EventHub state. /// public Dictionary Partitions { get; private set; } /// /// Current EventHub state using data time. /// public Dictionary PartitionsDateTime { get; private set; } /// /// Union of and . /// [JsonIgnore] public Dictionary PartitionsDetailed { get { // PartitionsDetailed return this.Partitions.Union(this.PartitionsDateTime.Select(kv => new KeyValuePair(kv.Key, kv.Value.ToString("u")))) .GroupBy(kv => kv.Key) .ToDictionary(kv => kv.Key, group => string.Join(";", group.Select(kv => kv.Value))); } } /// /// The models name (timestamp + name). /// public string ModelName { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/Operations/000077500000000000000000000000001323713302100225625ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/Operations/CheckPointOperation.cs000066400000000000000000000117351323713302100270300ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Microsoft.ApplicationInsights.DataContracts; using Microsoft.WindowsAzure.Storage.Blob; using Newtonsoft.Json; using System; using System.Collections.Generic; using System.Globalization; using System.IO; using System.Threading.Tasks; using VW.Azure.Trainer.Data; namespace VW.Azure.Trainer { internal partial class Learner { internal bool ShouldCheckpoint(int numExamples) { // don't checkpoint if we didn't see any valid events. return this.trackbackList.Count > 0 && this.settings.CheckpointPolicy.ShouldCheckpointAfterExample(numExamples); } internal CheckpointData CreateCheckpointData(bool updateClientModel) { // TODO: checkpoint resolver state. var data = new CheckpointData { Timestamp = DateTime.UtcNow.ToString("yyyyMMdd/HHmmss", CultureInfo.InvariantCulture), UpdateClientModel = updateClientModel, StartDateTime = this.startDateTime }; var modelId = Guid.NewGuid().ToString(); // store the model name this.state.ModelName = $"{data.Timestamp}/model"; data.State = JsonConvert.SerializeObject(this.State); data.TrackbackCount = this.trackbackList.Count; data.TrackbackList = $"modelid: {modelId}\n" + string.Join("\n", this.trackbackList); this.trackbackList.Clear(); using (var memStream = new MemoryStream()) { this.vw.ID = modelId; this.vw.SaveModel(memStream); data.Model = memStream.ToArray(); return data; } } internal async Task Checkpoint(object obj) { try { var data = obj as CheckpointData; if (data == null) { this.telemetry.TrackTrace($"Received invalid data: {data}"); return; } var modelName = data.Timestamp + "/model"; this.telemetry.TrackTrace( "CheckPoint " + modelName, SeverityLevel.Information); var container = this.blobClient.GetContainerReference(this.settings.StorageContainerName); await container.CreateIfNotExistsAsync(); // save model to storage account // save trackback file to storage account var trackbackName = string.Format(CultureInfo.InvariantCulture, "{0}/model.trackback", data.Timestamp); var trackbackBlob = container.GetBlockBlobReference(trackbackName); // keep a history of state files var stateName = string.Format(CultureInfo.InvariantCulture, "{0}/{1}", data.Timestamp, Learner.StateBlobName); var stateBlob = container.GetBlockBlobReference(stateName); await Task.WhenAll( trackbackBlob.UploadTextAsync(data.TrackbackList), stateBlob.UploadTextAsync(data.State)); var modelBlob = await ExportModel(container, data.Model, modelName, data.TrackbackCount); // update the fast recovery state file var latestState = container.GetBlockBlobReference(Learner.StateBlobName); await latestState.StartCopyAsync(stateBlob); if (data.UpdateClientModel) { // update latest model var latestModel = await this.GetLatestModelBlob(); await latestModel.StartCopyAsync(modelBlob); } } catch (Exception ex) { this.telemetry.TrackException(ex); } } private async Task ExportModel(CloudBlobContainer container, byte[] model, string modelName, int numExamples) { var modelBlob = container.GetBlockBlobReference(modelName); await modelBlob.UploadFromByteArrayAsync(model, 0, model.Length); this.telemetry.TrackTrace( $"Model Save {modelBlob.Uri}", SeverityLevel.Information, new Dictionary { { "Size", model.Length.ToString() }, { "Uri", modelBlob.Uri.ToString() }, { "Examples added", numExamples.ToString() } }); return modelBlob; } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/Operations/EvalOperation.cs000066400000000000000000000223131323713302100256620ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Microsoft.ApplicationInsights; using Microsoft.ServiceBus.Messaging; using Newtonsoft.Json; using System; using System.Collections.Generic; using System.Globalization; using System.Linq; using System.Reactive.Linq; using System.Text; using System.Threading.Tasks.Dataflow; using VW.Azure.Trainer.Data; namespace VW.Azure.Trainer.Operations { internal sealed class EvalData { internal EvalEventData Data { get; set; } internal string JSON { get; set; } internal string PartitionKey { get; set; } } /// /// Evaluation Event Data /// public sealed class EvalEventData { /// /// Policy name /// [JsonProperty("name")] public string Name { get; set; } /// /// Weighted cost /// [JsonProperty("weightedcost")] public float WeightedCost { get; set; } /// /// Importance weight /// [JsonProperty("importanceweight")] public float ImportanceWeight { get; set; } /// /// Timestamp /// [JsonProperty("timestamp")] public DateTime Timestamp { get; set; } /// /// Event Id /// [JsonProperty("eventid")] public string EventId { get; set; } } internal sealed class EvalOperation : IDisposable { private readonly EventHubClient evalEventHubClient; private TransformManyBlock evalBlock; private IDisposable evalBlockDisposable; private TelemetryClient telemetry; private PerformanceCounters performanceCounters; internal EvalOperation(OnlineTrainerSettingsInternal settings, PerformanceCounters performanceCounters) { this.performanceCounters = performanceCounters; this.telemetry = new TelemetryClient(); // evaluation pipeline this.evalEventHubClient = EventHubClient.CreateFromConnectionString(settings.EvalEventHubConnectionString); this.evalBlock = new TransformManyBlock( (Func>)this.OfflineEvaluate, new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = 4, BoundedCapacity = 1024 }); this.evalBlock.Completion.Trace(this.telemetry, "Stage 4 - Evaluation pipeline"); // batch output together to match EventHub throughput by maintaining maximum latency of 1 seconds this.evalBlockDisposable = this.evalBlock.AsObservable() .GroupBy(k => k.PartitionKey) .Select(g => g.Window(TimeSpan.FromSeconds(1)) .Select(w => w.Buffer(245 * 1024, e => Encoding.UTF8.GetByteCount(e.JSON))) .SelectMany(w => w) .Subscribe(this.UploadEvaluation)) .Publish() .Connect(); } internal ITargetBlock TargetBlock { get { return this.evalBlock; } } private List OfflineEvaluate(object trainerResultObj) { try { var trainerResult = trainerResultObj as TrainerResult; if (trainerResult == null) { this.telemetry.TrackTrace($"Received invalid data: trainerResult is null"); return new List(); } return this.OfflineEvaluateInternal(trainerResult) // insert event id & timestamp to enable data correlation .Select(e => { e.EventId = trainerResult.EventId; e.Timestamp = trainerResult.Timestamp; var ed = new EvalData { PartitionKey = trainerResult.PartitionKey, Data = e, JSON = JsonConvert.SerializeObject(e) }; return ed; }) .ToList(); } catch (Exception e) { this.telemetry.TrackException(e); return new List(); } } private IEnumerable OfflineEvaluateInternal(TrainerResult trainerResult) { this.performanceCounters.Stage4_Evaluation_PerSec.Increment(); this.performanceCounters.Stage4_Evaluation_Total.Increment(); if (trainerResult == null) { yield break; } if (trainerResult.Label == null) { this.telemetry.TrackTrace($"Received invalid data: trainerResult.Label is null"); yield break; } if (trainerResult.ProgressiveProbabilities == null) { this.telemetry.TrackTrace($"Received invalid data: trainerResult.Probabilities is null"); yield break; } var pi_a_x = trainerResult.ProgressiveProbabilities[trainerResult.Label.Action - 1]; var p_a_x = trainerResult.Label.Probability * (1 - trainerResult.ProbabilityOfDrop); // the latest one we're currently training yield return new EvalEventData { Name = "Latest Policy", // calcuate expectation under current randomized policy (using current exploration strategy) // VW action is 0-based, label Action is 1 based WeightedCost = (trainerResult.Label.Cost * pi_a_x) / p_a_x, ImportanceWeight = pi_a_x / p_a_x }; // the one currently running yield return new EvalEventData { Name = "Deployed Policy", WeightedCost = trainerResult.Label.Cost, ImportanceWeight = 1 // for deployed policy just use the observed cost }; // Default = choosing the action that's supplied by caller yield return new EvalEventData { Name = "Default Policy", WeightedCost = VowpalWabbitContextualBanditUtil.GetUnbiasedCost(trainerResult.Label.Action, (uint)1, trainerResult.Label.Cost, trainerResult.Label.Probability), ImportanceWeight = trainerResult.Label.Action == 1 ? 1 / (trainerResult.ObservedProbabilities[0] * (1 - trainerResult.ProbabilityOfDrop)) : 0 }; // per action tag policies for (int action = 1; action <= trainerResult.ProgressiveRanking.Length; action++) { string tag; if (!trainerResult.ActionsTags.TryGetValue(action, out tag)) tag = action.ToString(CultureInfo.InvariantCulture); var name = $"Constant Policy {tag}"; yield return new EvalEventData { Name = name, WeightedCost = VowpalWabbitContextualBanditUtil.GetUnbiasedCost(trainerResult.Label.Action, (uint)action, trainerResult.Label.Cost, trainerResult.Label.Probability), ImportanceWeight = trainerResult.Label.Action == action ? 1 / (trainerResult.ObservedProbabilities[action - 1] * (1 - trainerResult.ProbabilityOfDrop)) : 0 }; } } private void UploadEvaluation(IList batch) { try { this.performanceCounters.Stage4_Evaluation_Total.Increment(); this.performanceCounters.Stage4_Evaluation_BatchesPerSec.Increment(); // construct multi-line JSON // TODO: check on how we batch in client library, we should use the EventId var eventData = new EventData(Encoding.UTF8.GetBytes(string.Join("\n", batch.Select(b => b.JSON)))) { PartitionKey = batch.First().PartitionKey }; this.evalEventHubClient.Send(eventData); } catch (Exception e) { this.telemetry.TrackException(e); } } public void Dispose() { if (this.evalBlock != null) { this.evalBlock.Complete(); this.evalBlock.Completion.Wait(TimeSpan.FromMinutes(1)); this.evalBlock = null; } if (this.evalBlockDisposable != null) { this.evalBlockDisposable.Dispose(); this.evalBlockDisposable = null; } } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/Operations/InitializeOperation.cs000066400000000000000000000216651323713302100271050ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Microsoft.ApplicationInsights.DataContracts; using Microsoft.WindowsAzure.Storage.Blob; using Newtonsoft.Json; using System; using System.Collections.Generic; using System.Globalization; using System.IO; using System.Linq; using System.Runtime.Caching; using System.Text.RegularExpressions; using System.Threading.Tasks; using VW; using VW.Serializer; namespace VW.Azure.Trainer { internal partial class Learner { public async Task FindAndResumeFromState() { var container = this.blobClient.GetContainerReference(this.settings.StorageContainerName); await container.CreateIfNotExistsAsync(); var stateBlob = container.GetBlockBlobReference(Learner.StateBlobName); if (await stateBlob.ExistsAsync() && await this.TryResumeFromState(stateBlob)) return; // find days var dayDirectories = from c in container.ListBlobs(useFlatBlobListing: false) let dir = c as CloudBlobDirectory where dir != null && dir.Prefix.Length >= 8 let date = DateTime.ParseExact(dir.Prefix.Substring(0, 8), "yyyyMMdd", CultureInfo.InvariantCulture) orderby date descending select dir; foreach (var day in dayDirectories) { // find state files var states = from f in container.ListBlobs(day.Prefix, useFlatBlobListing: true) let file = f as CloudBlockBlob where file != null let match = Regex.Match(file.Name, @"^\d{8}/(\d{6})/" + Learner.StateBlobName + "$") where match.Success let time = DateTime.ParseExact(match.Groups[1].Value, "HHmmss", CultureInfo.InvariantCulture) orderby time descending select file; foreach (var file in states) { if (await TryResumeFromState(file)) return; } } // unable to find a model, fallback to fresh start this.FreshStart(); } internal void FreshStart(OnlineTrainerState state = null, byte[] model = null) { if (state == null) state = new OnlineTrainerState(); this.telemetry.TrackTrace("Fresh Start", SeverityLevel.Information); // start from scratch this.state = state; // save extra state so learning can be resumed later with new data var settings = new VowpalWabbitSettings("--save_resume --preserve_performance_counters " + this.settings.Metadata.TrainArguments); if (model != null) settings.ModelStream = new MemoryStream(model); this.InitializeVowpalWabbit(settings); } private async Task TryLoadModel() { // find the model blob if (string.IsNullOrEmpty(this.state.ModelName)) { this.telemetry.TrackTrace("Model not specified"); return false; } var container = this.blobClient.GetContainerReference(this.settings.StorageContainerName); if (!await container.ExistsAsync()) { this.telemetry.TrackTrace($"Storage container missing '{this.settings.StorageContainerName}'"); return false; } var modelBlob = container.GetBlockBlobReference(this.state.ModelName); if (!await modelBlob.ExistsAsync()) { this.telemetry.TrackTrace($"Model blob '{this.state.ModelName}' is missing"); return false; } // load the model var args = "--save_resume --preserve_performance_counters " + this.settings.Metadata.TrainArguments; try { using (var modelStream = await modelBlob.OpenReadAsync()) { // it's up to the external system to make sure the train arguments are compatible with the stored model // if the arguments are changed substantially, one needs to invoke Reset which forces a refresh this.InitializeVowpalWabbit(new VowpalWabbitSettings(args) { ModelStream = modelStream }); this.telemetry.TrackTrace($"Model loaded {this.state.ModelName}", SeverityLevel.Verbose); } } catch (VowpalWabbitArgumentDisagreementException ex) { // found conflicting arguments. Start fresh model this.InitializeVowpalWabbit(new VowpalWabbitSettings(args)); this.telemetry.TrackTrace($"Arguments found in model {this.state.ModelName} disagree with newly supplied arguments: {args}. Discarding model and starting fresh: {ex.Message}", SeverityLevel.Verbose); } // store the initial model this.settings.InitialVowpalWabbitModel = this.state.ModelName; return true; } private void InitializeVowpalWabbit(VowpalWabbitSettings vwSettings) { if (this.settings.EnableExampleTracing) { vwSettings.EnableStringExampleGeneration = true; vwSettings.EnableStringFloatCompact = true; } vwSettings.EnableThreadSafeExamplePooling = true; vwSettings.MaxExamples = 64 * 1024; try { this.startDateTime = DateTime.UtcNow; this.vw = new VW.VowpalWabbit(vwSettings); var cmdLine = vw.Arguments.CommandLine; if (!(cmdLine.Contains("--cb_explore") || cmdLine.Contains("--cb_explore_adf"))) throw new ArgumentException("Only cb_explore and cb_explore_adf are supported"); } catch (Exception ex) { this.telemetry.TrackException(ex, new Dictionary { { "help", "Invalid model. For help go to https://github.com/JohnLangford/vowpal_wabbit/wiki/Azure-Trainer" } }); throw ex; } this.referenceResolver = new VowpalWabbitJsonReferenceResolver( this.delayedExampleCallback, cacheRequestItemPolicyFactory: key => new CacheItemPolicy() { SlidingExpiration = TimeSpan.FromHours(1), RemovedCallback = this.CacheEntryRemovedCallback }); //this.vwAllReduce = new VowpalWabbitThreadedLearning(vwSettings.ShallowCopy( // maxExampleQueueLengthPerInstance: 4*1024, // parallelOptions: new ParallelOptions // { // MaxDegreeOfParallelism = 2, // }, // exampleDistribution: VowpalWabbitExampleDistribution.RoundRobin, // exampleCountPerRun: 128 * 1024)); } private void CacheEntryRemovedCallback(CacheEntryRemovedArguments arguments) { switch (arguments.RemovedReason) { case CacheEntryRemovedReason.Evicted: case CacheEntryRemovedReason.Expired: // free memory var serializer = (VowpalWabbitJsonSerializer)arguments.CacheItem.Value; serializer.Dispose(); this.perfCounters.Feature_Requests_Pending.IncrementBy(-1); this.perfCounters.Feature_Requests_Discarded.Increment(); break; } } private async Task TryResumeFromState(CloudBlockBlob stateBlob) { using (var stream = await stateBlob.OpenReadAsync()) using (var reader = new JsonTextReader(new StreamReader(stream))) { var jsonSerializer = JsonSerializer.CreateDefault(); this.state = jsonSerializer.Deserialize(reader); } this.telemetry.TrackTrace( $"Resume from '{stateBlob.Uri}'", SeverityLevel.Verbose, this.state.PartitionsDetailed); return await this.TryLoadModel(); } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/Operations/LatencyOperation.cs000066400000000000000000000017641323713302100264010ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Threading.Tasks; using VW.Azure.Trainer.Data; namespace VW.Azure.Trainer { internal sealed class LatencyOperation : ThrottledOperation { public LatencyOperation() : base(TimeSpan.FromSeconds(1)) { } protected override Task ProcessInternal(TrainerResult value) { this.telemetry.TrackMetric("End-to-End Latency " + value.PartitionId, value.Latency.TotalSeconds); return Task.FromResult(true); } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/Operations/LearnOperation.cs000066400000000000000000000116501323713302100260360ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Microsoft.ApplicationInsights.DataContracts; using System; using System.Collections.Generic; using System.Diagnostics; using System.Linq; using System.Reactive.Linq; using VW.Azure.Trainer.Data; using VW.Labels; namespace VW.Azure.Trainer { internal partial class Learner { public TrainerResult Learn(PipelineData example) { try { if (this.settings.EnableExampleTracing) this.telemetry.TrackTrace( "Example", SeverityLevel.Verbose, new Dictionary { { "ID", example.EventId }, { "VW", example.Example.VowpalWabbitString }, { "JSON", example.JSON } }); var label = example.Example.Labels .OfType() .FirstOrDefault(l => l.Probability != 0f || l.Cost != 0); if (label == null) this.telemetry.TrackTrace($"Unable to find valid label for event '{example.EventId}'", SeverityLevel.Warning); // predict first then learn to avoid information leak var progressivePrediction = example.Example.Learn(VowpalWabbitPredictionType.ActionProbabilities, this.vw); //if (this.vwAllReduce != null) //{ // this.vwAllReduce.Post(vw => // { // var actions = example.Example.Learn(VowpalWabbitPredictionType.Multilabel, vw); // PerformanceCounters.Instance.ExamplesLearnedTotal.Increment(); // PerformanceCounters.Instance.ExamplesLearnedSec.Increment(); // PerformanceCounters.Instance.FeaturesLearnedSec.IncrementBy((long)example.Example.NumberOfFeatures); // example.Example.Dispose(); // }); //} // record event id for reproducibility this.trackbackList.Add(example.EventId); this.perfCounters.Stage2_Learn_Total.Increment(); this.perfCounters.Stage2_Learn_ExamplesPerSec.Increment(); this.perfCounters.Stage2_Learn_FeaturesPerSec.IncrementBy((long)example.Example.NumberOfFeatures); // measure latency const int TimeSpanTicksPerMillisecond = 10000; var latency = DateTime.UtcNow - example.Timestamp; var performanceCounterTicks = latency.Ticks * Stopwatch.Frequency / TimeSpanTicksPerMillisecond; this.perfCounters.AverageExampleLatency.IncrementBy(performanceCounterTicks); this.perfCounters.AverageExampleLatencyBase.Increment(); // update partition state if (example.PartitionId != null && example.PartitionId != null) { this.state.Partitions[example.PartitionId] = example.Offset; // this.state.PartitionsDateTime[eventHubExample.PartitionKey] = eventHubExample.Offset; } return new TrainerResult(progressivePrediction, example.Actions, example.Probabilities) { Label = label, PartitionId = example.PartitionId, PartitionKey = example.PartitionKey, Latency = latency, ProbabilityOfDrop = example.ProbabilityOfDrop, ActionsTags = example.ActionsTags, EventId = example.EventId, Timestamp = example.Timestamp }; } catch (Exception ex) { this.telemetry.TrackException(ex, new Dictionary { { "ID", example.EventId }, { "VW", example.Example.VowpalWabbitString }, { "JSON", example.JSON } }); this.perfCounters.Stage2_Faulty_ExamplesPerSec.Increment(); this.perfCounters.Stage2_Faulty_Examples_Total.Increment(); return null; } finally { if (example.Example != null) example.Example.Dispose(); } } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/Operations/ThrottledOperation.cs000066400000000000000000000044031323713302100267440ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Microsoft.ApplicationInsights; using System; using System.Linq; using System.Reactive.Linq; using System.Reactive.Subjects; using System.Threading.Tasks; namespace VW.Azure.Trainer { internal abstract class ThrottledOperation : IDisposable { private Subject pipeline; private IDisposable pipelineDisposable; protected readonly TelemetryClient telemetry; protected ThrottledOperation(TimeSpan dueTime) { this.telemetry = new TelemetryClient(); this.pipeline = new Subject(); // limit the number of events to every 5 seconds var connectable = this.pipeline .Throttle(TimeSpan.FromSeconds(5)) .SelectMany(value => Observable.FromAsync(async () => { try { await this.ProcessInternal(value); } catch (Exception e) { this.telemetry.TrackException(e); } } )) .Replay(); this.pipelineDisposable = connectable.Connect(); } internal void Process(T performance) { this.pipeline.OnNext(performance); } protected abstract Task ProcessInternal(T value); public void Dispose() { if (this.pipeline != null) { this.pipeline.Dispose(); this.pipeline = null; } if (this.pipelineDisposable != null) { this.pipelineDisposable.Dispose(); this.pipelineDisposable = null; } } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/PerformanceCounters.cs000066400000000000000000000301561323713302100247570ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Microsoft.ApplicationInsights.Extensibility; using Microsoft.ApplicationInsights.Extensibility.PerfCounterCollector; using System; using System.Linq; using System.Diagnostics; using Microsoft.ApplicationInsights; using System.Collections.Generic; using System.Text.RegularExpressions; using System.Text; namespace VW.Azure.Trainer { /// /// Performance counters reporting various metrics. /// public sealed class PerformanceCounters : IDisposable { /// /// Performance counter attribute to autmatically allow counter creation. /// public class PerformanceCounterTypeAttribute : Attribute { /// /// Initializes a new instance. /// public PerformanceCounterTypeAttribute(PerformanceCounterType type, string name = null) { this.Type = type; this.Name = name; } /// /// The performance counter type. /// public PerformanceCounterType Type { get; private set; } /// /// The desired name for the performance counter. /// public string Name { get; private set; } } private const string category = "Online Trainer"; static PerformanceCounters() { try { if (PerformanceCounterCategory.Exists(category)) PerformanceCounterCategory.Delete(category); // order to be sure that *Base follows counter var props = typeof(PerformanceCounters) .GetProperties() .Where(p => p.PropertyType == typeof(PerformanceCounter)) .OrderBy(p => p.Name).ToList(); var counterCollection = new CounterCreationDataCollection(); foreach (var p in props) { var attr = (PerformanceCounterTypeAttribute)p.GetCustomAttributes(typeof(PerformanceCounterTypeAttribute), true).First(); counterCollection.Add(new CounterCreationData() { CounterName = p.Name, CounterHelp = string.Empty, CounterType = attr.Type }); } PerformanceCounterCategory.Create(category, "Online Trainer Perf Counters", PerformanceCounterCategoryType.MultiInstance, counterCollection); } catch (Exception e) { new TelemetryClient().TrackException(e); } } /// /// Initializes a new instance. /// public PerformanceCounters(string instance) { try { var perfCollectorModule = new PerformanceCollectorModule(); var props = typeof(PerformanceCounters) .GetProperties() .Where(p => p.PropertyType == typeof(PerformanceCounter)); var all = new List(); foreach (var p in props) { var counter = new PerformanceCounter(category, p.Name, instance, false); p.SetValue(this, counter); counter.RawValue = 0; all.Add(counter); if (!p.Name.EndsWith("Base", StringComparison.Ordinal)) { var perfCounterSpec = $"\\{category}({instance})\\{p.Name}"; var reportAs = p.Name .Replace('_', ' ') .Replace("Per", "/"); // http://i1.blogs.msdn.com/b/visualstudioalm/archive/2015/04/01/application-insights-choose-your-own-performance-counters.aspx // Currently, metric names may only contain letters, round brackets, forward slashes, hyphens, underscores, spaces and dots. var reportAsStringBuilder = new StringBuilder(reportAs); foreach (Match match in Regex.Matches(reportAs, "[0-9]")) reportAsStringBuilder[match.Index] = (char)('A' + (match.Groups[0].Value[0] - '0')); perfCollectorModule.Counters.Add(new PerformanceCounterCollectionRequest(perfCounterSpec, reportAsStringBuilder.ToString())); } } perfCollectorModule.Initialize(TelemetryConfiguration.Active); this.All = all.ToArray(); } catch (Exception e) { new TelemetryClient().TrackException(e); } } /// /// Disposes performance counter native resources. /// public void Dispose() { var props = typeof(PerformanceCounters) .GetProperties() .Where(p => p.PropertyType == typeof(IDisposable)); foreach (var p in props) { var perfCounter = (IDisposable)p.GetValue(this); if (perfCounter != null) { perfCounter.Dispose(); p.SetValue(this, null); } } } /// /// List of all online trainer performance counters. /// public PerformanceCounter[] All { get; private set; } /// /// Number of active Azure EventHub event processors. /// [PerformanceCounterType(PerformanceCounterType.NumberOfItems32)] public PerformanceCounter EventHub_Processors { get; private set; } /// /// Number of cached features. /// [PerformanceCounterType(PerformanceCounterType.NumberOfItems32)] public PerformanceCounter Features_Cached { get; private set; } /// /// Number of pending feature requests. Features referenced by ID, which have not yet occured in the stream. /// [PerformanceCounterType(PerformanceCounterType.NumberOfItems64)] public PerformanceCounter Feature_Requests_Pending { get; private set; } /// /// Number of feature requests discarded (e.g. due to timeout hit). /// [PerformanceCounterType(PerformanceCounterType.NumberOfItems64)] public PerformanceCounter Feature_Requests_Discarded { get; private set; } /// /// Total number of batches received by stage 0. /// [PerformanceCounterType(PerformanceCounterType.NumberOfItems64)] public PerformanceCounter Stage0_Batches_Total { get; private set; } /// /// Number of batches received per second by stage 0. /// [PerformanceCounterType(PerformanceCounterType.RateOfCountsPerSecond64)] public PerformanceCounter Stage0_BatchesPerSec { get; private set; } /// /// Average size of batches received per second by stage 0. /// [PerformanceCounterType(PerformanceCounterType.AverageCount64)] public PerformanceCounter Stage0_Batches_Size { get; private set; } /// /// Average (base) size of batches received per second by stage 0. /// [PerformanceCounterType(PerformanceCounterType.AverageBase)] public PerformanceCounter Stage0_Batches_SizeBase { get; private set; } /// /// Bytes/sec received by stage 0. /// [PerformanceCounterType(PerformanceCounterType.RateOfCountsPerSecond64)] public PerformanceCounter Stage0_IncomingBytesPerSec { get; private set; } /// /// Number of JSON lines in stage 1. /// [PerformanceCounterType(PerformanceCounterType.NumberOfItems64)] public PerformanceCounter Stage1_JSON_Queue { get; private set; } /// /// Number of JSON lines deserialized per second in stage 1. /// [PerformanceCounterType(PerformanceCounterType.RateOfCountsPerSecond64)] public PerformanceCounter Stage1_JSON_DeserializePerSec { get; private set; } /// /// Number of examples queued up for learning. /// [PerformanceCounterType(PerformanceCounterType.NumberOfItems64)] public PerformanceCounter Stage2_Learn_Queue { get; private set; } /// /// Total number of examples learned so far. /// [PerformanceCounterType(PerformanceCounterType.NumberOfItems64)] public PerformanceCounter Stage2_Learn_Total { get; private set; } /// /// Number of examples learned per second. /// [PerformanceCounterType(PerformanceCounterType.RateOfCountsPerSecond64)] public PerformanceCounter Stage2_Learn_ExamplesPerSec { get; private set; } /// /// Number of features learned per second. /// [PerformanceCounterType(PerformanceCounterType.RateOfCountsPerSecond64)] public PerformanceCounter Stage2_Learn_FeaturesPerSec { get; private set; } /// /// Total number of faulty examples encountered so far. /// [PerformanceCounterType(PerformanceCounterType.NumberOfItems64)] public PerformanceCounter Stage2_Faulty_Examples_Total { get; private set; } /// /// Number of faulty examples encountered per second. /// [PerformanceCounterType(PerformanceCounterType.RateOfCountsPerSecond64)] public PerformanceCounter Stage2_Faulty_ExamplesPerSec { get; private set; } /// /// Number of checkpoint requests queued up. /// [PerformanceCounterType(PerformanceCounterType.NumberOfItems64)] public PerformanceCounter Stage3_Checkpoint_Queue { get; private set; } /// /// Total number of evaluation outputs produced. /// [PerformanceCounterType(PerformanceCounterType.NumberOfItems64)] public PerformanceCounter Stage4_Evaluation_Total { get; private set; } /// /// Number of evaluation produced per second. /// [PerformanceCounterType(PerformanceCounterType.RateOfCountsPerSecond64)] public PerformanceCounter Stage4_Evaluation_PerSec { get; private set; } /// /// Total number of evaluation batches produces so far. /// [PerformanceCounterType(PerformanceCounterType.NumberOfItems64)] public PerformanceCounter Stage4_Evaluation_BatchesTotal { get; private set; } /// /// Number of evaluation batches produced per second. /// [PerformanceCounterType(PerformanceCounterType.RateOfCountsPerSecond64)] public PerformanceCounter Stage4_Evaluation_BatchesPerSec { get; private set; } /// /// Average example latency. /// [PerformanceCounterType(PerformanceCounterType.AverageTimer32)] public PerformanceCounter AverageExampleLatency { get; private set; } /// /// Average (base) example latency. /// [PerformanceCounterType(PerformanceCounterType.AverageBase)] public PerformanceCounter AverageExampleLatencyBase { get; private set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/Util/000077500000000000000000000000001323713302100213545ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/Util/ExtensionMethods.cs000066400000000000000000000041241323713302100252040ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Microsoft.ApplicationInsights; using System; using System.Collections.Generic; using System.Reactive.Linq; using System.Threading.Tasks; namespace VW.Azure.Trainer { internal static class ExtensionMethods { internal static Task Trace(this Task task, TelemetryClient telemetry, string message) { return task.ContinueWith(t => { telemetry.TrackTrace($"{message} completed: {t.Status}"); if (t.IsFaulted) telemetry.TrackException(t.Exception); }); } internal static IObservable> Buffer(this IObservable source, int maxSize, Func measure) { return Observable.Create>(obs => { var state = new List(); var size = 0; return source.Subscribe( onNext: v => { size += measure(v); state.Add(v); if (size >= maxSize) { obs.OnNext(state); state = new List(); size = 0; } }, onError: e => obs.OnError(e), onCompleted: () => { if (state.Count > 0) { obs.OnNext(state); } obs.OnCompleted(); }); }); } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/Trainer/Util/SafeTimer.cs000066400000000000000000000033241323713302100235640ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Threading; using System.Threading.Tasks; namespace VW.Azure.Trainer { internal class SafeTimer { private CancellationTokenSource cancellationTokenSource; private ManualResetEventSlim finishedEvent; internal SafeTimer(TimeSpan delay, Action action) { this.cancellationTokenSource = new CancellationTokenSource(); var cancellationToken = cancellationTokenSource.Token; finishedEvent = new ManualResetEventSlim(); Task.Factory .StartNew(async () => { while (true) { cancellationToken.ThrowIfCancellationRequested(); action(); cancellationToken.ThrowIfCancellationRequested(); await Task.Delay(delay, cancellationToken); } }, TaskCreationOptions.LongRunning) .ContinueWith(t => finishedEvent.Set()); } public void Stop(TimeSpan timeout) { cancellationTokenSource.Cancel(); finishedEvent.Wait(timeout); } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure/app.config000066400000000000000000000150631323713302100210070ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/azure/azure.csproj000066400000000000000000000366121323713302100214130ustar00rootroot00000000000000 Debug AnyCPU 8.0.30703 2.0 {43E32C1D-21D6-4BE3-8FA5-D5BA379CBA53} Library Properties VowpalWabbit.Azure VowpalWabbit.Azure v4.6 512 ..\ true true full false bin\Debug\ DEBUG;TRACE prompt 4 x64 pdbonly true bin\Release\ TRACE prompt 4 true true $(SolutionDir)\x64\Debug\ DEBUG;TRACE full x64 prompt MinimumRecommendedRules.ruleset true $(SolutionDir)\x64\Debug\VowpalWabbit.Azure.xml $(SolutionDir)\x64\Release\ TRACE true pdbonly x64 prompt MinimumRecommendedRules.ruleset true $(SolutionDir)\x64\Release\VowpalWabbit.Azure.xml true ..\vw_key.snk $(SolutionDir)\packages\gitlink.2.3.0\lib\net45\GitLink.exe True $(SolutionDir)\packages\Microsoft.ApplicationInsights.Agent.Intercept.2.0.6\lib\net45\Microsoft.AI.Agent.Intercept.dll True $(SolutionDir)\packages\Microsoft.ApplicationInsights.DependencyCollector.2.2.0\lib\net45\Microsoft.AI.DependencyCollector.dll True $(SolutionDir)\packages\Microsoft.ApplicationInsights.PerfCounterCollector.2.2.0\lib\net45\Microsoft.AI.PerfCounterCollector.dll True $(SolutionDir)\packages\Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.2.2.0\lib\net45\Microsoft.AI.ServerTelemetryChannel.dll True $(SolutionDir)\packages\Microsoft.ApplicationInsights.WindowsServer.2.2.0\lib\net45\Microsoft.AI.WindowsServer.dll True $(SolutionDir)\packages\Microsoft.ApplicationInsights.2.2.0\lib\net46\Microsoft.ApplicationInsights.dll True $(SolutionDir)\packages\Microsoft.ApplicationInsights.TraceListener.2.2.0\lib\net45\Microsoft.ApplicationInsights.TraceListener.dll True $(SolutionDir)\packages\Microsoft.Azure.KeyVault.Core.2.0.4\lib\net45\Microsoft.Azure.KeyVault.Core.dll True $(SolutionDir)\packages\Microsoft.Data.Edm.5.8.1\lib\net40\Microsoft.Data.Edm.dll True $(SolutionDir)\packages\Microsoft.Data.OData.5.8.1\lib\net40\Microsoft.Data.OData.dll True $(SolutionDir)\packages\Microsoft.Data.Services.Client.5.8.1\lib\net40\Microsoft.Data.Services.Client.dll True $(SolutionDir)\packages\EnterpriseLibrary.TransientFaultHandling.6.0.1304.0\lib\portable-net45+win+wp8\Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.dll True $(SolutionDir)\packages\CommonServiceLocator.1.3\lib\portable-net4+sl5+netcore45+wpa81+wp8\Microsoft.Practices.ServiceLocation.dll True $(SolutionDir)\packages\WindowsAzure.ServiceBus.3.4.3\lib\net45-full\Microsoft.ServiceBus.dll True $(SolutionDir)\packages\Microsoft.Azure.ServiceBus.EventProcessorHost.2.2.8\lib\net45-full\Microsoft.ServiceBus.Messaging.EventProcessorHost.dll True True False $(SolutionDir)\packages\WindowsAzure.Storage.7.2.1\lib\net40\Microsoft.WindowsAzure.Storage.dll True $(SolutionDir)\packages\Newtonsoft.Json.9.0.1\lib\net45\Newtonsoft.Json.dll True $(SolutionDir)\packages\System.Reactive.Core.3.1.1\lib\net46\System.Reactive.Core.dll True $(SolutionDir)\packages\System.Reactive.Interfaces.3.1.1\lib\net45\System.Reactive.Interfaces.dll True $(SolutionDir)\packages\System.Reactive.Linq.3.1.1\lib\net46\System.Reactive.Linq.dll True $(SolutionDir)\packages\System.Reactive.PlatformServices.3.1.1\lib\net46\System.Reactive.PlatformServices.dll True $(SolutionDir)\packages\System.Reactive.Windows.Threading.3.1.1\lib\net45\System.Reactive.Windows.Threading.dll True $(SolutionDir)\packages\System.Spatial.5.8.1\lib\net40\System.Spatial.dll True $(SolutionDir)\packages\Microsoft.Tpl.Dataflow.4.5.24\lib\portable-net45+win8+wpa81\System.Threading.Tasks.Dataflow.dll True Designer Designer {85e55ae0-3784-4968-9271-c81af560e1c1} vw_clr {e621e022-c1f8-433f-905a-ab9a3de072b7} vw_common {e4e962ae-7056-4eb0-a8c5-8dc824a4b068} cs {9e27fa94-ab34-4736-8427-fb7a2ba90d52} cs_json false vowpal-wabbit-8.5.0.dfsg1/cs/azure/azure.nuspec000066400000000000000000000062041323713302100214020ustar00rootroot00000000000000 Vowpal Wabbit Azure VowpalWabbit.Azure vw vowpal wabbit langford ml machine learning azure John Langford et al $version$ https://github.com/JohnLangford/vowpal_wabbit/wiki/C%23-Binding https://github.com/JohnLangford/vowpal_wabbit/blob/master/LICENSE false Official Vowpal Wabbit library including C# interface Copyright (C) Microsoft Corp 2012-2016, Yahoo! Inc. 2007-2012, and many individual contributors. All rights reserved. vowpal-wabbit-8.5.0.dfsg1/cs/azure/packages.config000066400000000000000000000045361323713302100220100ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/azure_service/000077500000000000000000000000001323713302100205535ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/azure_service/DownloadServiceConfiguration.ps1000066400000000000000000000004601323713302100270200ustar00rootroot00000000000000$mcURL = Read-Host -Prompt "Management Center URL" $storageAccountKey = Read-Host "Storage Account Key" $storageAccountKey = [uri]::EscapeDataString($storageAccountKey) Invoke-WebRequest -Uri "$($mcURL)Deployment/GenerateTrainerConfig?key=$storageAccountKey" -OutFile ServiceConfiguration.Local.cscfg vowpal-wabbit-8.5.0.dfsg1/cs/azure_service/OnlineTrainerContent/000077500000000000000000000000001323713302100246575ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/azure_service/OnlineTrainerContent/GCSettingsManagement.ps1000066400000000000000000000171721323713302100313230ustar00rootroot00000000000000<# //********************************************************* // // Copyright (c) Microsoft. All rights reserved. // This code is licensed under the Microsoft Public License. // THIS CODE IS PROVIDED *AS IS* WITHOUT WARRANTY OF // ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING ANY // IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR // PURPOSE, MERCHANTABILITY, OR NON-INFRINGEMENT. // //********************************************************* #> Param ( $serverGC = $True, $backgroundGC = $True ) [string]$configFilePath = "$(${env:RoleRoot})\base\x64\WaWorkerHost.exe.config" function Create-ConfigFileIfNotExists { # Only create the Xml document if it does not already exist if(-not (Test-Path -Path $configFilePath -PathType Leaf)) { [System.Xml.XmlDocument]$document = New-Object System.Xml.XmlDocument # config file doesn't exist create a now one [System.Xml.XmlDeclaration]$prolog = $document.CreateXmlDeclaration("1.0", "utf-8", $null) [System.Xml.XmlNode]$child = $document.AppendChild($prolog) [System.Xml.XmlElement]$configurationElement = Append-ElementIfNotExists $document $document.DocumentElement "configuration" # Save a copy of the document $document.Save($configFilePath) } } function Load-ConfigFile { [System.Xml.XmlDocument]$document = New-Object System.Xml.XmlDocument #Check if the document already exists and load it if it does not if(Test-Path -Path $configFilePath -PathType Leaf) { $document.Load($configFilePath) } return $document } function Append-ElementIfNotExists { param ( [System.Xml.XmlDocument]$document, [System.Xml.XmlElement]$parent, [string]$elementName ) [System.Xml.XmlElement]$element = $null [System.Xml.XmlNode]$parentNode = $parent if($document -ne $null) { if($parentNode -eq $null) { $parentNode = $document } $element = $parentNode.SelectSingleNode("./$($elementName)") if($element -eq $null) { $element = $document.CreateElement($elementName) [System.Xml.XmlElement]$child = $parentNode.AppendChild($element) } } return $element } function Create-ElementStructureIfNotExists { param ( [System.Xml.XmlDocument]$document ) [bool]$isSuccess = $false if($document -ne $null) { [System.Xml.XmlElement]$configurationElement = Append-ElementIfNotExists $document $null "configuration" if($configurationElement -ne $null) { [System.Xml.XmlElement]$element = Append-ElementIfNotExists $document $configurationElement "runtime" $isSuccess = $element -ne $null } } return $isSuccess } # Create the document if required Create-ConfigFileIfNotExists # Load the configuration file into the XML document [System.Xml.XmlDocument]$configurationDocument = Load-ConfigFile if($configurationDocument -ne $null) { if(Create-ElementStructureIfNotExists $configurationDocument) { # All of the entries are on the runtime element [System.Xml.XmlElement]$runtimeElement = $configurationDocument.DocumentElement.SelectSingleNode('./runtime') if($runtimeElement -ne $null) { # Set the Server GC to enabled if requested [System.Xml.XmlElement]$serverGCElement = Append-ElementIfNotExists $configurationDocument $runtimeElement "gcServer" $serverGCElement.SetAttribute("enabled", $serverGC.ToString([System.Globalization.CultureInfo]::InvariantCulture).ToLower()) # Set the concurrent GC to enabled if requested [System.Xml.XmlElement]$concurrentGCElement = Append-ElementIfNotExists $configurationDocument $runtimeElement "gcConcurrent" $concurrentGCElement.SetAttribute("enabled", $backgroundGC.ToString([System.Globalization.CultureInfo]::InvariantCulture).ToLower()) } } # Save the document $configurationDocument.Save($configFilePath) } vowpal-wabbit-8.5.0.dfsg1/cs/azure_service/OnlineTrainerContent/ServerGC.cmd000066400000000000000000000020251323713302100270230ustar00rootroot00000000000000 REM ********************************************************* REM REM Copyright (c) Microsoft. All rights reserved. REM This code is licensed under the Microsoft Public License. REM THIS CODE IS PROVIDED *AS IS* WITHOUT WARRANTY OF REM ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING ANY REM IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR REM PURPOSE, MERCHANTABILITY, OR NON-INFRINGEMENT. REM REM ********************************************************* REM Check if the script is running in the Azure emulator and if so do not run IF "%IsEmulated%"=="true" goto :EOF If "%UseServerGC%"=="False" GOTO :ValidateBackground If "%UseServerGC%"=="0" GOTO :ValidateBackground SET UseServerGC="True" :ValidateBackground If "%UseBackgroundGC%"=="False" GOTO :CommandExecution If "%UseBackgroundGC%"=="0" GOTO :CommandExecution SET UseBackgroundGC="True" :CommandExecution PowerShell.exe -executionpolicy unrestricted -command ".\GCSettingsManagement.ps1" -serverGC %UseServerGC% -backgroundGC %UseBackgroundGC% Exit /bvowpal-wabbit-8.5.0.dfsg1/cs/azure_service/OnlineTrainerContent/diagnostics.wadcfgx000066400000000000000000000060271323713302100305400ustar00rootroot00000000000000 true vowpal-wabbit-8.5.0.dfsg1/cs/azure_service/OnlineTrainerContent/install.cmd000066400000000000000000000054271323713302100270220ustar00rootroot00000000000000REM Set the value of netfx to install appropriate .NET Framework. REM ***** To install .NET 4.5.2 set the variable netfx to "NDP452" ***** REM ***** To install .NET 4.6 set the variable netfx to "NDP46" ***** REM ***** To install .NET 4.6.1 set the variable netfx to "NDP461" ***** set netfx="NDP461" REM ***** Set script start timestamp ***** set timehour=%time:~0,2% set timestamp=%date:~-4,4%%date:~-10,2%%date:~-7,2%-%timehour: =0%%time:~3,2% set "log=install.cmd started %timestamp%." REM ***** Exit script if running in Emulator ***** if %ComputeEmulatorRunning%=="true" goto exit REM ***** Needed to correctly install .NET 4.6.1, otherwise you may see an out of disk space error ***** set TMP=%PathToNETFXInstall% set TEMP=%PathToNETFXInstall% REM ***** Setup .NET filenames and registry keys ***** if %netfx%=="NDP461" goto NDP461 if %netfx%=="NDP46" goto NDP46 set "netfxinstallfile=NDP452-KB2901954-Web.exe" set netfxregkey="0x5cbf5" goto logtimestamp :NDP46 set "netfxinstallfile=NDP46-KB3045560-Web.exe" set netfxregkey="0x60051" goto logtimestamp :NDP461 set "netfxinstallfile=NDP461-KB3102438-Web.exe" set netfxregkey="0x6041f" :logtimestamp REM ***** Setup LogFile with timestamp ***** md "%PathToNETFXInstall%\log" set startuptasklog="%PathToNETFXInstall%log\startuptasklog-%timestamp%.txt" set netfxinstallerlog="%PathToNETFXInstall%log\NetFXInstallerLog-%timestamp%" echo %log% >> %startuptasklog% echo Logfile generated at: %startuptasklog% >> %startuptasklog% echo TMP set to: %TMP% >> %startuptasklog% echo TEMP set to: %TEMP% >> %startuptasklog% REM ***** Check if .NET is installed ***** echo Checking if .NET (%netfx%) is installed >> %startuptasklog% reg query "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\Full" /v Release | Find %netfxregkey% if %ERRORLEVEL%== 0 goto installed REM ***** Installing .NET ***** echo Installing .NET with commandline: start /wait %~dp0%netfxinstallfile% /q /serialdownload /log %netfxinstallerlog% /chainingpackage "CloudService Startup Task" >> %startuptasklog% start /wait %~dp0%netfxinstallfile% /q /serialdownload /log %netfxinstallerlog% /chainingpackage "CloudService Startup Task" >> %startuptasklog% 2>>&1 if %ERRORLEVEL%== 0 goto installed echo .NET installer exited with code %ERRORLEVEL% >> %startuptasklog% if %ERRORLEVEL%== 3010 goto restart if %ERRORLEVEL%== 1641 goto restart echo .NET (%netfx%) install failed with Error Code %ERRORLEVEL%. Further logs can be found in %netfxinstallerlog% >> %startuptasklog% :restart echo Restarting to complete .NET (%netfx%) installation >> %startuptasklog% goto end :installed echo .NET (%netfx%) is installed >> %startuptasklog% :end echo install.cmd completed: %date:~-4,4%%date:~-10,2%%date:~-7,2%-%timehour: =0%%time:~3,2% >> %startuptasklog% :exit EXIT /B 0vowpal-wabbit-8.5.0.dfsg1/cs/azure_service/RoleProperties.txt000066400000000000000000000001041323713302100242650ustar00rootroot00000000000000EntryPoint=VowpalWabbit.AzureWorker.dll TargetFrameWorkVersion=v4.0 vowpal-wabbit-8.5.0.dfsg1/cs/azure_service/ServiceConfiguration.Cloud.cscfg000066400000000000000000000031221323713302100267550ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/azure_service/ServiceConfiguration.Local.cscfg000066400000000000000000000031151323713302100267430ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/azure_service/ServiceConfiguration.cscfg000066400000000000000000000020431323713302100257110ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/azure_service/ServiceDefinition.csdef000066400000000000000000000043131323713302100251730ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/azure_service/azure_service.ccproj000066400000000000000000000075121323713302100246300ustar00rootroot00000000000000 Debug AnyCPU 2.9 7263f536-3abe-4a8e-ba66-1341d6a53579 Library Properties OnlineTraining OnlineTraining True azure_service False true full false bin\Debug\ DEBUG;TRACE prompt 4 pdbonly true bin\Release\ TRACE prompt 4 OnlineTrainer {9DE727D8-FD1B-4144-8976-5371663D4C22} True Worker OnlineTrainer True Content Content Content Content 10.0 $(MSBuildExtensionsPath)\Microsoft\VisualStudio\v$(VisualStudioVersion)\Windows Azure Tools\2.8\ powershell -Command "if (-not (Test-Path '$(ProjectDir)\OnlineTrainerContent\NDP461-KB3102438-Web.exe')){ Invoke-WebRequest -Uri 'https://download.microsoft.com/download/3/5/9/35980F81-60F4-4DE3-88FC-8F962B97253B/NDP461-KB3102438-Web.exe' -OutFile $(ProjectDir)\OnlineTrainerContent\NDP461-KB3102438-Web.exe }" vowpal-wabbit-8.5.0.dfsg1/cs/azure_worker/000077500000000000000000000000001323713302100204245ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/azure_worker/CheckpointController.cs000066400000000000000000000023551323713302100251130ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System.Net; using System.Net.Http; using System.Threading.Tasks; using VW.Azure.Trainer; namespace VW.Azure.Worker { /// /// HTTP Front end to trigger checkpointing /// public sealed class CheckpointController : OnlineTrainerController { public CheckpointController(LearnEventProcessorHost trainProcessorFactory) : base(trainProcessorFactory) { } public async Task Get() { if (!this.TryAuthorize()) return this.Request.CreateResponse(HttpStatusCode.Unauthorized); await this.trainProcessorHost.CheckpointAsync(); return this.Request.CreateResponse(HttpStatusCode.OK); } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure_worker/OnlineTrainerController.cs000066400000000000000000000026771323713302100256040ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Microsoft.ApplicationInsights; using Microsoft.Azure; using System.Linq; using System.Web.Http; using VW.Azure.Trainer; namespace VW.Azure.Worker { public class OnlineTrainerController : ApiController { // injected through Unity protected readonly LearnEventProcessorHost trainProcessorHost; protected readonly string adminToken; protected readonly TelemetryClient telemetry; public OnlineTrainerController(LearnEventProcessorHost trainProcessorFactory) { this.telemetry = new TelemetryClient(); this.trainProcessorHost = trainProcessorFactory; this.adminToken = CloudConfigurationManager.GetSetting("AdminToken"); } protected bool TryAuthorize() { var header = this.Request.Headers.SingleOrDefault(x => x.Key == "Authorization"); return !(header.Value == null || adminToken != header.Value.First()); } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure_worker/OnlineTrainerSettingsDownloader.cs000066400000000000000000000121021323713302100272600ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Microsoft.Azure; using Microsoft.WindowsAzure.ServiceRuntime; using Microsoft.WindowsAzure.Storage; using Microsoft.WindowsAzure.Storage.Blob; using System; using System.Diagnostics; using System.IO; using System.Linq; using System.Net; using System.Reactive.Linq; using System.Threading; using System.Threading.Tasks; using VW.Azure.Trainer; namespace VW.Azure.Worker { internal sealed class OnlineTrainerSettingsDownloader : IDisposable { internal static CloudBlob GetSettingsBlockBlob() { var storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString")); var blobClient = storageAccount.CreateCloudBlobClient(); var container = blobClient.GetContainerReference(OnlineTrainerSettings.SettingsContainerName); return container.GetBlobReference(OnlineTrainerSettings.LatestClientSettingsBlobName); } public delegate void DownloadedEventHandler(object sender, byte[] data); public delegate void FailedEventHandler(object sender, Exception e); public event DownloadedEventHandler Downloaded; public event FailedEventHandler Failed; private IDisposable disposable; private CloudBlob settingsBlob; private string blobEtag; public OnlineTrainerSettingsDownloader(TimeSpan interval) { this.settingsBlob = GetSettingsBlockBlob(); RoleEnvironment.Changed += RoleEnvironment_Changed; // run background thread var conn = Observable.Interval(interval) .SelectMany(_ => Observable.FromAsync(this.Execute)) .Replay(); this.disposable = conn.Connect(); } private void RoleEnvironment_Changed(object sender, RoleEnvironmentChangedEventArgs e) { var change = e.Changes .OfType() .FirstOrDefault(c => c.ConfigurationSettingName == "StorageConnectionString"); if (change != null) this.settingsBlob = GetSettingsBlockBlob(); } private async Task Execute(CancellationToken cancellationToken) { var uri = string.Empty; try { uri = this.settingsBlob.Uri.ToString(); // avoid not found exception if (!await this.settingsBlob.ExistsAsync(cancellationToken)) return; if (this.settingsBlob.Properties != null) { // if downloadImmediately is set to false, the downloader // will not download the blob on first check, and on second check // onwards, the blob must have changed before a download is triggered. // this is to support caller who manually downloads the blob first for // other purposes and do not want to redownload. // avoid not modified exception if (this.settingsBlob.Properties.ETag == this.blobEtag) return; var currentBlobEtag = this.blobEtag; this.blobEtag = this.settingsBlob.Properties.ETag; } // download using (var ms = new MemoryStream()) { await this.settingsBlob.DownloadToStreamAsync(ms, cancellationToken); Trace.TraceInformation("Retrieved new blob for {0}", this.settingsBlob.Uri); this.Downloaded?.Invoke(this, ms.ToArray()); } } catch (Exception ex) { if (ex is StorageException) { RequestResult result = ((StorageException)ex).RequestInformation; if (result.HttpStatusCode != (int)HttpStatusCode.NotFound) { Trace.TraceError( "Failed to retrieve '{0}': {1}. {2}", uri, ex.Message, result.HttpStatusMessage); } } else Trace.TraceError("Failed to retrieve '{0}': {1}", uri, ex.Message); this.Failed?.Invoke(this, ex); } } public void Dispose() { RoleEnvironment.Changed -= RoleEnvironment_Changed; if (this.disposable != null) { this.disposable.Dispose(); this.disposable = null; } } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure_worker/OnlineTrainerSettingsWatcher.cs000066400000000000000000000172031323713302100265660ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Microsoft.ApplicationInsights; using Microsoft.ApplicationInsights.DataContracts; using Microsoft.Azure; using Microsoft.ServiceBus; using Microsoft.WindowsAzure.ServiceRuntime; using Microsoft.WindowsAzure.Storage; using Newtonsoft.Json; using System; using System.Collections.Generic; using System.Globalization; using System.IO; using System.Linq; using System.Net; using System.Text; using System.Threading.Tasks; using VW.Azure.Trainer; using VW.Azure.Trainer.Checkpoint; namespace VW.Azure.Worker { internal sealed class OnlineTrainerSettingsWatcher : IDisposable { private readonly TelemetryClient telemetry; private readonly LearnEventProcessorHost trainProcessorHost; private OnlineTrainerSettingsDownloader settingsDownloader; private OnlineTrainerSettings metaData; internal OnlineTrainerSettingsWatcher(LearnEventProcessorHost trainProcessorHost) { this.telemetry = new TelemetryClient(); this.trainProcessorHost = trainProcessorHost; RoleEnvironment.Changed += RoleEnvironment_Changed; this.settingsDownloader = new OnlineTrainerSettingsDownloader(TimeSpan.FromSeconds(5)); this.settingsDownloader.Downloaded += AzureSettingsBlobDownloader_Downloaded; this.settingsDownloader.Failed += AzureSettingsBlobDownloader_Failed; } public void RestartTrainProcessorHost() { if (this.metaData == null) return; var settings = new OnlineTrainerSettingsInternal { StorageConnectionString = CloudConfigurationManager.GetSetting("StorageConnectionString"), JoinedEventHubConnectionString = CloudConfigurationManager.GetSetting("JoinedEventHubConnectionString"), EvalEventHubConnectionString = CloudConfigurationManager.GetSetting("EvalEventHubConnectionString"), Metadata = this.metaData, CheckpointPolicy = ParseCheckpointPolicy(), // make sure we ignore previous events EventHubStartDateTimeUtc = DateTime.UtcNow }; var joinedEventHubConsumerGroup = CloudConfigurationManager.GetSetting("JoinedEventHubConsumerGroup"); if (!string.IsNullOrEmpty(joinedEventHubConsumerGroup)) settings.JoinedEventHubConsumerGroup = joinedEventHubConsumerGroup; bool enableExampleTracing; if (bool.TryParse(CloudConfigurationManager.GetSetting("EnableExampleTracing"), out enableExampleTracing)) settings.EnableExampleTracing = enableExampleTracing; ServiceBusConnectionStringBuilder serviceBusConnectionStringBuilder; try { serviceBusConnectionStringBuilder = new ServiceBusConnectionStringBuilder(settings.JoinedEventHubConnectionString); } catch (Exception e) { throw new InvalidDataException($"Invalid JoinedEventHubConnectionString '{settings.JoinedEventHubConnectionString}' found in settings: {e.Message}"); } try { serviceBusConnectionStringBuilder = new ServiceBusConnectionStringBuilder(settings.EvalEventHubConnectionString); } catch (Exception e) { throw new InvalidDataException($"Invalid EventHubEvalConnectionString '{settings.EvalEventHubConnectionString}' found in settings: {e.Message}"); } if (string.IsNullOrEmpty(serviceBusConnectionStringBuilder.EntityPath)) throw new InvalidDataException($"Invalid EventHubEvalConnectionString '{settings.EvalEventHubConnectionString}' found in settings: EntityPath missing"); CloudStorageAccount cloudStorageAccount; if (!CloudStorageAccount.TryParse(settings.StorageConnectionString, out cloudStorageAccount)) throw new InvalidDataException($"Invalid StorageConnectionString '{settings.StorageConnectionString}' found in settings"); // fire and forget var task = this.trainProcessorHost.Restart(settings); } private void AzureSettingsBlobDownloader_Failed(object sender, Exception e) { this.telemetry.TrackException(e); } private void AzureSettingsBlobDownloader_Downloaded(object sender, byte[] data) { try { var json = Encoding.UTF8.GetString(data); this.metaData = JsonConvert.DeserializeObject(json); this.telemetry.TrackTrace( "Metadata update. Trigger restart", SeverityLevel.Information, new Dictionary { { "settings", json } }); this.RestartTrainProcessorHost(); } catch (Exception ex) { this.telemetry.TrackException(ex); } } private void RoleEnvironment_Changed(object sender, RoleEnvironmentChangedEventArgs e) { try { var changes = e.Changes.OfType() .Select(c => c.ConfigurationSettingName) .ToList(); if (changes.Count > 0) { this.telemetry.TrackTrace( "Configuration changes. Trigger restart", SeverityLevel.Information, changes.ToDictionary(name => name, name => CloudConfigurationManager.GetSetting(name))); // fire and forget this.RestartTrainProcessorHost(); } } catch (Exception ex) { this.telemetry.TrackException(ex); } } private ICheckpointPolicy ParseCheckpointPolicy() { var checkpointString = CloudConfigurationManager.GetSetting("CheckpointIntervalOrCount"); if (!string.IsNullOrWhiteSpace(checkpointString)) { if (checkpointString.Contains(":")) { TimeSpan interval; if (TimeSpan.TryParse(checkpointString, CultureInfo.InvariantCulture, out interval)) return new IntervalCheckpointPolicy(interval); } else { int syncCount; if (int.TryParse(checkpointString, NumberStyles.Integer, CultureInfo.InvariantCulture, out syncCount)) return new CountingCheckpointPolicy(syncCount); } } this.telemetry.TrackTrace("No valid checkpoint policy found. Defaulting to 5 minute wallclock checkpointing."); return new IntervalCheckpointPolicy(TimeSpan.FromMinutes(5)); } public void Dispose() { RoleEnvironment.Changed -= RoleEnvironment_Changed; if (this.settingsDownloader != null) { this.settingsDownloader.Dispose(); this.settingsDownloader = null; } } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure_worker/Properties/000077500000000000000000000000001323713302100225605ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/azure_worker/Properties/AssemblyInfo.cs000066400000000000000000000022131323713302100255000ustar00rootroot00000000000000//------------------------------------------------------------------------------ // // This code was generated by a tool. // Runtime Version:4.0.30319.42000 // // Changes to this file may cause incorrect behavior and will be lost if // the code is regenerated. // //------------------------------------------------------------------------------ [assembly: System.Reflection.AssemblyTitle("Vowpal Wabbit")] [assembly: System.Reflection.AssemblyDescription("Vowpal Wabbit")] [assembly: System.Reflection.AssemblyCompany("Microsoft Corp")] [assembly: System.Reflection.AssemblyProduct("Vowpal Wabbit")] [assembly: System.Reflection.AssemblyCopyright("Copyright (C) Microsoft Corp 2012-2016, Yahoo! Inc. 2007-2012, and many individua" + "l contributors. All rights reserved")] [assembly: System.Runtime.InteropServices.ComVisible(false)] [assembly: System.CLSCompliant(false)] [assembly: System.Runtime.InteropServices.Guid("6a577997-af00-4ca0-8453-fdc8bbdf2a57")] [assembly: System.Reflection.AssemblyVersion("8.4.0.1")] [assembly: System.Reflection.AssemblyFileVersion("8.4.0.1")] vowpal-wabbit-8.5.0.dfsg1/cs/azure_worker/ResetController.cs000066400000000000000000000065161323713302100241110ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Newtonsoft.Json; using System; using System.Collections.Generic; using System.Linq; using System.Net; using System.Net.Http; using System.Text; using System.Threading.Tasks; using System.Web.Http; using VW.Azure.Trainer; namespace VW.Azure.Worker { public sealed class ResetController : OnlineTrainerController { public ResetController(LearnEventProcessorHost trainProcessorFactory) : base(trainProcessorFactory) { } /// /// Vanilla reset. /// [HttpGet] public async Task Get() { if (!this.TryAuthorize()) return this.Request.CreateResponse(HttpStatusCode.Unauthorized); try { await this.trainProcessorHost.ResetModelAsync(); return this.Request.CreateResponse(HttpStatusCode.OK); } catch (Exception ex) { this.telemetry.TrackException(ex); return this.Request.CreateResponse(HttpStatusCode.InternalServerError, ex.Message); } } /// /// Reset optionally include EventHub position. /// [HttpPost] public async Task Post() { if (!this.TryAuthorize()) return this.Request.CreateResponse(HttpStatusCode.Unauthorized); try { OnlineTrainerState state = null; var body = await Request.Content.ReadAsStringAsync(); if (!string.IsNullOrWhiteSpace(body)) state = JsonConvert.DeserializeObject(body); await this.trainProcessorHost.ResetModelAsync(state); return this.Request.CreateResponse(HttpStatusCode.OK); } catch (Exception ex) { this.telemetry.TrackException(ex); return this.Request.CreateResponse(HttpStatusCode.InternalServerError, ex.Message); } } /// /// Reset including a warm started model. /// [HttpPut] public async Task Put() { if (!this.TryAuthorize()) return this.Request.CreateResponse(HttpStatusCode.Unauthorized); try { var model = await Request.Content.ReadAsByteArrayAsync(); await this.trainProcessorHost.ResetModelAsync(model: model); return this.Request.CreateResponse(HttpStatusCode.OK); } catch (Exception ex) { this.telemetry.TrackException(ex); return this.Request.CreateResponse(HttpStatusCode.InternalServerError, ex.Message); } } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure_worker/StatusController.cs000066400000000000000000000027171323713302100243110ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Newtonsoft.Json.Linq; using System.Linq; using System.Web.Http; using VW.Azure.Trainer; namespace VW.Azure.Worker { /// /// HTTP Front end to expose performance statistics. /// public class StatusController : ApiController { private LearnEventProcessorHost trainProcessorHost; public StatusController(LearnEventProcessorHost trainProcessorHost) { this.trainProcessorHost = trainProcessorHost; } public IHttpActionResult Get() { var perfCounts = this.trainProcessorHost.PerformanceCounters; if (perfCounts == null) return Json(new { Message = "Not yet initialized." }); var status = new JObject(perfCounts.All.Select(pc => new JProperty(pc.CounterName, pc.RawValue))); status.Add(new JProperty("LastStartDateTimeUtc", this.trainProcessorHost.LastStartDateTimeUtc)); return Json(status); } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure_worker/WorkerRole.cs000066400000000000000000000130431323713302100230470ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Microsoft.ApplicationInsights; using Microsoft.ApplicationInsights.DataContracts; using Microsoft.ApplicationInsights.Extensibility; using Microsoft.Azure; using Microsoft.Owin.Cors; using Microsoft.Owin.Hosting; using Microsoft.Practices.Unity; using Microsoft.WindowsAzure.ServiceRuntime; using Owin; using System; using System.Diagnostics; using System.Net; using System.Threading; using System.Web.Http; using System.Web.Http.Cors; using Unity.WebApi; using VW.Azure.Trainer; namespace VW.Azure.Worker { public class WorkerRole : RoleEntryPoint { private readonly ManualResetEventSlim stopEvent; private IDisposable webApp; private TelemetryClient telemetry; private OnlineTrainerSettingsWatcher settingsWatcher; private LearnEventProcessorHost trainProcesserHost; public WorkerRole() { this.stopEvent = new ManualResetEventSlim(); } public override bool OnStart() { try { // Set the maximum number of concurrent connections ServicePointManager.DefaultConnectionLimit = 128; // For information on handling configuration changes // see the MSDN topic at http://go.microsoft.com/fwlink/?LinkId=166357. bool result = base.OnStart(); TelemetryConfiguration.Active.InstrumentationKey = CloudConfigurationManager.GetSetting("APPINSIGHTS_INSTRUMENTATIONKEY"); //TelemetryConfiguration.Active.TelemetryChannel.DeveloperMode = true; this.telemetry = new TelemetryClient(); try { this.telemetry.TrackTrace("WorkerRole starting", SeverityLevel.Information); this.trainProcesserHost = new LearnEventProcessorHost(); this.settingsWatcher = new OnlineTrainerSettingsWatcher(this.trainProcesserHost); this.StartRESTAdminEndpoint(); } catch (Exception e) { this.telemetry.TrackException(e); // still start to give AppInsights a chance to log } return result; } catch (Exception e) { Debugger.Log(1, "ERROR", $"VowpalWabbit.AzureWorker failed to start: {e.Message} {e.StackTrace}"); throw; } } public override void Run() { try { // wait for OnStop this.stopEvent.Wait(); } catch (Exception e) { this.telemetry.TrackException(e); } } private void StartRESTAdminEndpoint() { // setup REST endpoint var endpoint = RoleEnvironment.CurrentRoleInstance.InstanceEndpoints["OnlineTrainer"]; string baseUri = String.Format("{0}://{1}", endpoint.Protocol, endpoint.IPEndpoint); this.webApp = WebApp.Start(baseUri, app => { var container = new UnityContainer(); // Register controller container.RegisterType(); container.RegisterType(); container.RegisterType(); // Register interface container.RegisterInstance(typeof(LearnEventProcessorHost), this.trainProcesserHost); var config = new HttpConfiguration(); config.DependencyResolver = new UnityDependencyResolver(container); config.EnableCors(new EnableCorsAttribute("*", "*", "*")); config.Routes.MapHttpRoute( "Default", "{controller}/{id}", new { id = RouteParameter.Optional }); // config.Services.Add(typeof(IExceptionLogger), new AiWebApiExceptionLogger()); app.UseCors(CorsOptions.AllowAll); app.UseWebApi(config); }); } public override void OnStop() { this.telemetry.TrackTrace("WorkerRole stopping", SeverityLevel.Information); try { this.stopEvent.Set(); if (this.settingsWatcher != null) { this.settingsWatcher.Dispose(); this.settingsWatcher = null; } if (this.trainProcesserHost != null) { this.trainProcesserHost.Dispose(); this.trainProcesserHost = null; } if (this.webApp != null) { this.webApp.Dispose(); this.webApp = null; } base.OnStop(); } catch (Exception e) { this.telemetry.TrackException(e); } this.telemetry.TrackTrace("WorkerRole stopped", SeverityLevel.Information); } } } vowpal-wabbit-8.5.0.dfsg1/cs/azure_worker/app.config000066400000000000000000000141701323713302100223760ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/azure_worker/azure_worker.csproj000066400000000000000000000455071323713302100244000ustar00rootroot00000000000000 Debug x64 {9DE727D8-FD1B-4144-8976-5371663D4C22} Library Properties VowpalWabbit.AzureWorker VowpalWabbit.AzureWorker Worker v4.6 512 ..\..\vowpalwabbit\ true ..\..\vowpalwabbit\x64\DebugRole\ DEBUG;TRACE full x64 prompt MinimumRecommendedRules.ruleset ..\..\vowpalwabbit\x64\ReleaseRole\ TRACE true pdbonly x64 prompt MinimumRecommendedRules.ruleset true ..\vw_key.snk $(SolutionDir)\packages\Microsoft.ApplicationInsights.Agent.Intercept.2.0.6\lib\net45\Microsoft.AI.Agent.Intercept.dll True $(SolutionDir)\packages\Microsoft.ApplicationInsights.DependencyCollector.2.2.0\lib\net45\Microsoft.AI.DependencyCollector.dll True $(SolutionDir)\packages\Microsoft.ApplicationInsights.PerfCounterCollector.2.2.0\lib\net45\Microsoft.AI.PerfCounterCollector.dll True $(SolutionDir)\packages\Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.2.2.0\lib\net45\Microsoft.AI.ServerTelemetryChannel.dll True $(SolutionDir)\packages\Microsoft.ApplicationInsights.Web.2.2.0\lib\net45\Microsoft.AI.Web.dll True $(SolutionDir)\packages\Microsoft.ApplicationInsights.WindowsServer.2.2.0\lib\net45\Microsoft.AI.WindowsServer.dll True $(SolutionDir)\packages\Microsoft.ApplicationInsights.2.2.0\lib\net46\Microsoft.ApplicationInsights.dll True ..\..\vowpalwabbit\packages\Microsoft.ApplicationInsights.TraceListener.2.2.0\lib\net45\Microsoft.ApplicationInsights.TraceListener.dll True $(SolutionDir)\packages\Microsoft.Azure.KeyVault.Core.2.0.4\lib\net45\Microsoft.Azure.KeyVault.Core.dll True $(SolutionDir)\packages\Microsoft.Data.Edm.5.8.1\lib\net40\Microsoft.Data.Edm.dll True $(SolutionDir)\packages\Microsoft.Data.OData.5.8.1\lib\net40\Microsoft.Data.OData.dll True $(SolutionDir)\packages\Microsoft.Data.Services.Client.5.8.1\lib\net40\Microsoft.Data.Services.Client.dll True $(SolutionDir)\packages\Microsoft.Owin.3.0.1\lib\net45\Microsoft.Owin.dll True $(SolutionDir)\packages\Microsoft.Owin.Cors.3.0.1\lib\net45\Microsoft.Owin.Cors.dll True $(SolutionDir)\packages\Microsoft.Owin.Host.HttpListener.3.0.1\lib\net45\Microsoft.Owin.Host.HttpListener.dll True $(SolutionDir)\packages\Microsoft.Owin.Hosting.3.0.1\lib\net45\Microsoft.Owin.Hosting.dll True $(SolutionDir)\packages\EnterpriseLibrary.TransientFaultHandling.6.0.1304.0\lib\portable-net45+win+wp8\Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.dll True $(SolutionDir)\packages\CommonServiceLocator.1.3\lib\portable-net4+sl5+netcore45+wpa81+wp8\Microsoft.Practices.ServiceLocation.dll True $(SolutionDir)\packages\Unity.4.0.1\lib\net45\Microsoft.Practices.Unity.dll True $(SolutionDir)\packages\Unity.4.0.1\lib\net45\Microsoft.Practices.Unity.Configuration.dll True $(SolutionDir)\packages\Unity.4.0.1\lib\net45\Microsoft.Practices.Unity.RegistrationByConvention.dll True $(SolutionDir)\packages\WindowsAzure.ServiceBus.3.4.3\lib\net45-full\Microsoft.ServiceBus.dll True $(SolutionDir)\packages\Microsoft.Azure.ServiceBus.EventProcessorHost.2.2.8\lib\net45-full\Microsoft.ServiceBus.Messaging.EventProcessorHost.dll True $(SolutionDir)\packages\Microsoft.WindowsAzure.ConfigurationManager.3.2.3\lib\net40\Microsoft.WindowsAzure.Configuration.dll True $(SolutionDir)\packages\WindowsAzure.Storage.7.2.1\lib\net40\Microsoft.WindowsAzure.Storage.dll True $(SolutionDir)\packages\Newtonsoft.Json.9.0.1\lib\net45\Newtonsoft.Json.dll True $(SolutionDir)\packages\Owin.1.0\lib\net40\Owin.dll True $(SolutionDir)\packages\Microsoft.AspNet.WebApi.Client.5.2.3\lib\net45\System.Net.Http.Formatting.dll True $(SolutionDir)\packages\System.Reactive.Core.3.1.1\lib\net46\System.Reactive.Core.dll True $(SolutionDir)\packages\System.Reactive.Interfaces.3.1.1\lib\net45\System.Reactive.Interfaces.dll True $(SolutionDir)\packages\System.Reactive.Linq.3.1.1\lib\net46\System.Reactive.Linq.dll True $(SolutionDir)\packages\System.Reactive.PlatformServices.3.1.1\lib\net46\System.Reactive.PlatformServices.dll True $(SolutionDir)\packages\System.Reactive.Windows.Threading.3.1.1\lib\net45\System.Reactive.Windows.Threading.dll True $(SolutionDir)\packages\System.Spatial.5.8.1\lib\net40\System.Spatial.dll True $(SolutionDir)\packages\Microsoft.Tpl.Dataflow.4.5.24\lib\portable-net45+win8+wpa81\System.Threading.Tasks.Dataflow.dll True $(SolutionDir)\packages\Microsoft.AspNet.Cors.5.2.3\lib\net45\System.Web.Cors.dll True $(SolutionDir)\packages\Microsoft.AspNet.WebApi.Core.5.2.3\lib\net45\System.Web.Http.dll True $(SolutionDir)\packages\Microsoft.AspNet.WebApi.Cors.5.2.3\lib\net45\System.Web.Http.Cors.dll True $(SolutionDir)\packages\Microsoft.AspNet.WebApi.Owin.5.2.3\lib\net45\System.Web.Http.Owin.dll True $(SolutionDir)\packages\Unity.WebAPI.5.2.3\lib\net45\Unity.WebApi.dll True {43e32c1d-21d6-4be3-8fa5-d5ba379cba53} azure {85e55ae0-3784-4968-9271-c81af560e1c1} vw_clr {e621e022-c1f8-433f-905a-ab9a3de072b7} vw_common {e4e962ae-7056-4eb0-a8c5-8dc824a4b068} cs {9e27fa94-ab34-4736-8427-fb7a2ba90d52} cs_json vowpal-wabbit-8.5.0.dfsg1/cs/azure_worker/packages.config000066400000000000000000000072541323713302100234010ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/cli/000077500000000000000000000000001323713302100164545ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/cli/AssemblyInfo.cpp000066400000000000000000000014611323713302100215550ustar00rootroot00000000000000using namespace System; using namespace System::Reflection; using namespace System::Runtime::CompilerServices; using namespace System::Runtime::InteropServices; using namespace System::Security::Permissions; [assembly:AssemblyTitleAttribute("Vowpal Wabbit")]; [assembly:AssemblyDescriptionAttribute("")]; [assembly:AssemblyConfigurationAttribute("")]; [assembly:AssemblyCompanyAttribute("Microsoft Corp")]; [assembly:AssemblyProductAttribute("Vowpal Wabbit")]; [assembly:AssemblyCopyrightAttribute("Copyright (C) 2014")]; [assembly:AssemblyTrademarkAttribute("Copyright (C) Microsoft Corp 2012-2016, Yahoo! Inc. 2007-2012, and many individual contributors. All rights reserved")]; [assembly:AssemblyCultureAttribute("")]; [assembly:AssemblyVersionAttribute("8.4.0.1")]; [assembly:AssemblyFileVersion("8.4.0.1")]; vowpal-wabbit-8.5.0.dfsg1/cs/cli/Resource.rc000066400000000000000000000113661323713302100206000ustar00rootroot00000000000000// Microsoft Visual C++ generated resource script. // #include "resource.h" #define APSTUDIO_READONLY_SYMBOLS ///////////////////////////////////////////////////////////////////////////// // // Generated from the TEXTINCLUDE 2 resource. // #include "winres.h" ///////////////////////////////////////////////////////////////////////////// #undef APSTUDIO_READONLY_SYMBOLS ///////////////////////////////////////////////////////////////////////////// // English (United States) resources #if !defined(AFX_RESOURCE_DLL) || defined(AFX_TARG_ENU) LANGUAGE LANG_ENGLISH, SUBLANG_ENGLISH_US #pragma code_page(1252) #ifdef APSTUDIO_INVOKED ///////////////////////////////////////////////////////////////////////////// // // TEXTINCLUDE // 1 TEXTINCLUDE BEGIN "resource.h\0" END 2 TEXTINCLUDE BEGIN "#include ""winres.h""\r\n" "\0" END 3 TEXTINCLUDE BEGIN "\r\n" "\0" END #endif // APSTUDIO_INVOKED ///////////////////////////////////////////////////////////////////////////// // // Version // VS_VERSION_INFO VERSIONINFO FILEVERSION 8,4,0,1 PRODUCTVERSION 8,4,0,1 FILEFLAGSMASK 0x3fL #ifdef _DEBUG FILEFLAGS 0x1L #else FILEFLAGS 0x0L #endif FILEOS 0x40004L FILETYPE 0x2L FILESUBTYPE 0x0L BEGIN BLOCK "StringFileInfo" BEGIN BLOCK "040904b0" BEGIN VALUE "CompanyName", "Microsoft Corp" VALUE "FileDescription", "Vowpal Wabbit" VALUE "FileVersion", "8.4.0.1" VALUE "InternalName", "VowpalWabbit.Core.dll" VALUE "LegalCopyright", "Copyright (C) Microsoft Corp 2012-2016, Yahoo! Inc. 2007-2012, and many individual contributors. All rights reserved" VALUE "OriginalFilename", "VowpalWabbit.Core.dll" VALUE "ProductName", "Vowpal Wabbit" VALUE "ProductVersion", "8.4.0.1" END END BLOCK "VarFileInfo" BEGIN VALUE "Translation", 0x409, 1200 END END #endif // English (United States) resources ///////////////////////////////////////////////////////////////////////////// #ifndef APSTUDIO_INVOKED ///////////////////////////////////////////////////////////////////////////// // // Generated from the TEXTINCLUDE 3 resource. // ///////////////////////////////////////////////////////////////////////////// #endif // not APSTUDIO_INVOKED vowpal-wabbit-8.5.0.dfsg1/cs/cli/clr_io.cpp000066400000000000000000000032241323713302100204300ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #include #include "clr_io.h" using namespace System; using namespace System::Runtime::InteropServices; namespace VW { clr_io_buf::clr_io_buf(Stream^ stream) : m_stream(stream), m_buffer(nullptr) { if (stream == nullptr) throw gcnew ArgumentNullException("stream"); files.push_back(0); } void clr_io_buf::ensure_buffer_size(size_t nbytes) { if (m_buffer != nullptr && m_buffer->Length >= nbytes) return; m_buffer = gcnew array((int)nbytes); } int clr_io_buf::open_file(const char* name, bool stdin_off, int flag) { return 0; } void clr_io_buf::reset_file(int f) { m_stream->Seek(0, SeekOrigin::Begin); head = space.begin(); space.end() = space.begin(); } ssize_t clr_io_buf::read_file(int f, void* buf, size_t nbytes) { ensure_buffer_size(nbytes); int readBytes = m_stream->Read(m_buffer, 0, (int)nbytes); Marshal::Copy(m_buffer, 0, IntPtr(buf), (int)nbytes); return readBytes; } size_t clr_io_buf::num_files() { return 1; } ssize_t clr_io_buf::write_file(int file, const void* buf, size_t nbytes) { ensure_buffer_size(nbytes); Marshal::Copy(IntPtr((void*)buf), m_buffer, 0, (int)nbytes); m_stream->Write(m_buffer, 0, (int)nbytes); return nbytes; } bool clr_io_buf::compressed() { return false; } void clr_io_buf::flush() { io_buf::flush(); m_stream->Flush(); } bool clr_io_buf::close_file() { // don't close stream on purpose. Caller of SaveModel should have control when to close. return true; } } vowpal-wabbit-8.5.0.dfsg1/cs/cli/clr_io.h000066400000000000000000000020011323713302100200650ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #include "io_buf.h" using namespace System; using namespace System::IO; namespace VW { /// /// C++ wrapper for managed . /// class clr_io_buf : public io_buf { private: gcroot m_stream; gcroot^> m_buffer; void ensure_buffer_size(size_t nbytes); public: /// /// Initializes a new instance. /// clr_io_buf(Stream^ stream); virtual int open_file(const char* name, bool stdin_off, int flag = READ); virtual void reset_file(int f); virtual ssize_t read_file(int f, void* buf, size_t nbytes); virtual size_t num_files(); virtual ssize_t write_file(int file, const void* buf, size_t nbytes); virtual bool compressed(); virtual void flush(); virtual bool close_file(); }; } vowpal-wabbit-8.5.0.dfsg1/cs/cli/clr_io_memory.cpp000066400000000000000000000022141323713302100220160ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #include "clr_io_memory.h" namespace VW { clr_io_memory_buf::clr_io_memory_buf() { files.push_back(0); m_iterator = m_data.begin(); } int clr_io_memory_buf::open_file(const char* name, bool stdin_off, int flag) { m_iterator = m_data.begin(); return 0; } void clr_io_memory_buf::reset_file(int f) { size_t count = m_data.size(); m_iterator = m_data.begin(); } ssize_t clr_io_memory_buf::read_file(int f, void* buf, size_t nbytes) { size_t left_over = min(nbytes, m_data.end() - m_iterator); if (left_over == 0) return 0; memcpy_s(buf, nbytes, &*m_iterator, left_over); m_iterator += left_over; return left_over; } size_t clr_io_memory_buf::num_files() { return 1; } ssize_t clr_io_memory_buf::write_file(int file, const void* buf, size_t nbytes) { m_data.insert(m_data.end(), (char*)buf, (char*)buf + nbytes); return nbytes; } bool clr_io_memory_buf::compressed() { return false; } bool clr_io_memory_buf::close_file() { return true; } } vowpal-wabbit-8.5.0.dfsg1/cs/cli/clr_io_memory.h000066400000000000000000000016661323713302100214750ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #include "io_buf.h" #include namespace VW { /// /// IO Buffer keeping data in memory. Used by VowpalWabbit::Reload. /// class clr_io_memory_buf : public io_buf { private: std::vector m_data; std::vector::const_iterator m_iterator; public: /// /// Initializes a new instance. /// clr_io_memory_buf(); virtual int open_file(const char* name, bool stdin_off, int flag = READ); virtual void reset_file(int f); virtual ssize_t read_file(int f, void* buf, size_t nbytes); virtual size_t num_files(); virtual ssize_t write_file(int file, const void* buf, size_t nbytes); virtual bool compressed(); virtual bool close_file(); }; } vowpal-wabbit-8.5.0.dfsg1/cs/cli/cs_vld.vcxproj000066400000000000000000000207321323713302100213470ustar00rootroot00000000000000 DebugLeakCheck x64 Debug x64 Release x64 {3D57A6AF-DE8C-40DC-ABDE-E4CE1B9C0D20} v4.5.2 ManagedCProj cs_vld ..\vowpalwabbit\ ..\..\sdl\SDL-7.0-Recommended.ruleset true DynamicLibrary true v140 true Unicode DynamicLibrary true v140 true Unicode true DynamicLibrary false v140 true Unicode true $(Platform)\$(Configuration)\$(ProjectName)\ true $(Platform)\$(Configuration)\$(ProjectName)\ false $(Platform)\$(Configuration)\$(ProjectName)\ Level3 Disabled WIN32;_DEBUG;%(PreprocessorDefinitions) NotUsing %(AdditionalIncludeDirectories) true %(AdditionalLibraryDirectories) %(AdditionalManifestFiles) Level3 Disabled LEAKCHECK;WIN32;_DEBUG;%(PreprocessorDefinitions) NotUsing %(AdditionalIncludeDirectories) true %(AdditionalLibraryDirectories) %(AdditionalManifestFiles) Level3 WIN32;NDEBUG;%(PreprocessorDefinitions) NotUsing %(AdditionalIncludeDirectories) true %(AdditionalLibraryDirectories) This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. vowpal-wabbit-8.5.0.dfsg1/cs/cli/cs_vld.vcxproj.filters000066400000000000000000000021441323713302100230130ustar00rootroot00000000000000 {4FC737F1-C7A5-4376-A066-2A32D752A2FF} cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx {93995380-89BD-4b04-88EB-625FBE52EBFB} h;hh;hpp;hxx;hm;inl;inc;xsd {67DA6AB6-F800-4c08-8B7A-83BB121AAD01} rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx;tiff;tif;png;wav;mfcribbon-ms Header Files Source Files vowpal-wabbit-8.5.0.dfsg1/cs/cli/packages.config000066400000000000000000000003731323713302100214240ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/cli/resource.h000066400000000000000000000006031323713302100204530ustar00rootroot00000000000000//{{NO_DEPENDENCIES}} // Microsoft Visual C++ generated include file. // Used by Resource.rc // Next default values for new objects // #ifdef APSTUDIO_INVOKED #ifndef APSTUDIO_READONLY_SYMBOLS #define _APS_NEXT_RESOURCE_VALUE 101 #define _APS_NEXT_COMMAND_VALUE 40001 #define _APS_NEXT_CONTROL_VALUE 1001 #define _APS_NEXT_SYMED_VALUE 101 #endif #endif vowpal-wabbit-8.5.0.dfsg1/cs/cli/spanning_tree_clr.cpp000066400000000000000000000013021323713302100226500ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #include "spanning_tree_clr.h" #include "vw_clr.h" using namespace std; namespace VW { SpanningTreeClr::SpanningTreeClr() { m_spanningTree = new SpanningTree; } SpanningTreeClr::~SpanningTreeClr() { try { delete m_spanningTree; } CATCHRETHROW } void SpanningTreeClr::Start() { try { m_spanningTree->Start(); } CATCHRETHROW } void SpanningTreeClr::Stop() { try { m_spanningTree->Stop(); } CATCHRETHROW } void SpanningTreeClr::Run() { try { m_spanningTree->Run(); } CATCHRETHROW } } vowpal-wabbit-8.5.0.dfsg1/cs/cli/spanning_tree_clr.h000066400000000000000000000015231323713302100223220ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #pragma once #include "spanning_tree.h" namespace VW { /// /// Managed wrapper for AllReduce spanning tree implementation. /// public ref class SpanningTreeClr { private: SpanningTree* m_spanningTree; public: /// /// Initializes a new instance. /// SpanningTreeClr(); ~SpanningTreeClr(); /// /// Starts the server on a background thread. /// void Start(); /// /// Runs the server on the calling thread. /// void Run(); /// /// Stops the background thread. /// void Stop(); }; } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vld_clr.cpp000066400000000000000000000022561323713302100206120ustar00rootroot00000000000000#include "vld_clr.h" namespace VLD { int VldReportHook(int reportType, wchar_t *message, int *returnValue) { auto msg = gcnew String(message); System::Diagnostics::Debug::Write(msg); if (VisualLeakDetector::Instance) VisualLeakDetector::Instance->ReportInternal(reportType, msg); *returnValue = 0; /* don't debug break */ return 1; /* handled */ } VisualLeakDetector::VisualLeakDetector() : m_messages(gcnew List^>) { if (Instance != nullptr) { throw gcnew NotSupportedException("Only a single instance is supported."); } Instance = this; VLDSetReportHook(VLD_RPTHOOK_INSTALL, VldReportHook); } VisualLeakDetector::~VisualLeakDetector() { this->!VisualLeakDetector(); } VisualLeakDetector::!VisualLeakDetector() { VLDSetReportHook(VLD_RPTHOOK_REMOVE, VldReportHook); Instance = nullptr; } void VisualLeakDetector::ReportInternal(int reportType, String^ msg) { m_messages->Add(Tuple::Create(reportType, msg)); } void VisualLeakDetector::ReportLeaks() { VLDReportLeaks(); } List^>^ VisualLeakDetector::Messages::get() { return m_messages; } void VisualLeakDetector::MarkAllLeaksAsReported() { VLDMarkAllLeaksAsReported(); } } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vld_clr.h000066400000000000000000000012641323713302100202550ustar00rootroot00000000000000#include #include using namespace System; using namespace System::Collections::Generic; using namespace System::Runtime::InteropServices; namespace VLD { int VldReportHook(int reportType, wchar_t *message, int *returnValue); public ref class VisualLeakDetector { private: initonly List^>^ m_messages; !VisualLeakDetector(); public: VisualLeakDetector(); ~VisualLeakDetector(); static VisualLeakDetector^ Instance; void ReportInternal(int reportType, String^ msg); property List^>^ Messages { List^>^ get(); } void ReportLeaks(); void MarkAllLeaksAsReported(); }; } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vowpalwabbit.cpp000066400000000000000000000475601323713302100216750ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #include "vw_clr.h" #include "vowpalwabbit.h" #include "best_constant.h" #include "parser.h" #include "hash.h" #include "vw_example.h" #include "vw_allreduce.h" #include "vw_builder.h" #include "clr_io.h" #include "lda_core.h" #include "parse_example.h" #include "parse_example_json.h" using namespace System; using namespace System::Collections::Generic; using namespace System::Text; namespace VW { VowpalWabbit::VowpalWabbit(VowpalWabbitSettings^ settings) : VowpalWabbitBase(settings) { if (settings == nullptr) { throw gcnew ArgumentNullException("settings"); } if (settings->ParallelOptions != nullptr) { m_vw->all_reduce_type = AllReduceType::Thread; auto total = settings->ParallelOptions->MaxDegreeOfParallelism; if (settings->Root == nullptr) { m_vw->all_reduce = new AllReduceThreads(total, settings->Node); } else { auto parent_all_reduce = (AllReduceThreads*)settings->Root->m_vw->all_reduce; m_vw->all_reduce = new AllReduceThreads(parent_all_reduce, total, settings->Node); } } try { m_hasher = GetHasher(); } CATCHRETHROW } VowpalWabbit::VowpalWabbit(String^ args) : VowpalWabbit(gcnew VowpalWabbitSettings(args)) { } void VowpalWabbit::Driver() { try { LEARNER::generic_driver(*m_vw); } CATCHRETHROW } void VowpalWabbit::RunMultiPass() { if (m_vw->numpasses > 1) { try { adjust_used_index(*m_vw); m_vw->do_reset_source = true; VW::start_parser(*m_vw); LEARNER::generic_driver(*m_vw); VW::end_parser(*m_vw); } CATCHRETHROW } } VowpalWabbitPerformanceStatistics^ VowpalWabbit::PerformanceStatistics::get() { // see parse_args.cc:finish(...) auto stats = gcnew VowpalWabbitPerformanceStatistics(); if (m_vw->current_pass == 0) { stats->NumberOfExamplesPerPass = m_vw->sd->example_number; } else { stats->NumberOfExamplesPerPass = m_vw->sd->example_number / m_vw->current_pass; } stats->WeightedExampleSum = m_vw->sd->weighted_examples(); stats->WeightedLabelSum = m_vw->sd->weighted_labels; if (m_vw->holdout_set_off) if (m_vw->sd->weighted_labeled_examples > 0) stats->AverageLoss = m_vw->sd->sum_loss / m_vw->sd->weighted_labeled_examples; else stats->AverageLoss = System::Double::NaN; else if ((m_vw->sd->holdout_best_loss == FLT_MAX) || (m_vw->sd->holdout_best_loss == FLT_MAX * 0.5)) stats->AverageLoss = System::Double::NaN; else stats->AverageLoss = m_vw->sd->holdout_best_loss; float best_constant; float best_constant_loss; if (get_best_constant(*m_vw, best_constant, best_constant_loss)) { stats->BestConstant = best_constant; if (best_constant_loss != FLT_MIN) { stats->BestConstantLoss = best_constant_loss; } } stats->TotalNumberOfFeatures = m_vw->sd->total_features; return stats; } uint64_t VowpalWabbit::HashSpace(String^ s) { auto newHash = m_hasher(s, hash_base); #ifdef _DEBUG auto oldHash = HashSpaceNative(s); assert(newHash == oldHash); #endif return (uint32_t)newHash; } uint64_t VowpalWabbit::HashFeature(String^ s, size_t u) { auto newHash = m_hasher(s, u) & m_vw->parse_mask; #ifdef _DEBUG auto oldHash = HashFeatureNative(s, u); assert(newHash == oldHash); #endif return (uint64_t)newHash; } uint64_t VowpalWabbit::HashSpaceNative(String^ s) { auto bytes = System::Text::Encoding::UTF8->GetBytes(s); auto handle = GCHandle::Alloc(bytes, GCHandleType::Pinned); try { return VW::hash_space(*m_vw, reinterpret_cast(handle.AddrOfPinnedObject().ToPointer())); } CATCHRETHROW finally { handle.Free(); } } uint64_t VowpalWabbit::HashFeatureNative(String^ s, uint64_t u) { auto bytes = System::Text::Encoding::UTF8->GetBytes(s); auto handle = GCHandle::Alloc(bytes, GCHandleType::Pinned); try { return VW::hash_feature(*m_vw, reinterpret_cast(handle.AddrOfPinnedObject().ToPointer()), u); } CATCHRETHROW finally { handle.Free(); } } void VowpalWabbit::Learn(VowpalWabbitExample^ ex) { #if _DEBUG if (ex == nullptr) { throw gcnew ArgumentNullException("ex"); } #endif try { m_vw->learn(ex->m_example); // as this is not a ring-based example it is not free'd m_vw->l->finish_example(*m_vw, *ex->m_example); } CATCHRETHROW } generic T VowpalWabbit::Learn(VowpalWabbitExample^ ex, IVowpalWabbitPredictionFactory^ predictionFactory) { #if _DEBUG if (ex == nullptr) throw gcnew ArgumentNullException("ex"); if (nullptr == predictionFactory) throw gcnew ArgumentNullException("predictionFactory"); #endif try { m_vw->learn(ex->m_example); auto prediction = predictionFactory->Create(m_vw, ex->m_example); // as this is not a ring-based example it is not free'd m_vw->l->finish_example(*m_vw, *ex->m_example); return prediction; } CATCHRETHROW } void VowpalWabbit::Predict(VowpalWabbitExample^ ex) { #if _DEBUG if (ex == nullptr) throw gcnew ArgumentNullException("ex"); #endif try { m_vw->l->predict(*ex->m_example); // as this is not a ring-based example it is not free'd m_vw->l->finish_example(*m_vw, *ex->m_example); } CATCHRETHROW } generic T VowpalWabbit::Predict(VowpalWabbitExample^ ex, IVowpalWabbitPredictionFactory^ predictionFactory) { #if _DEBUG if (ex == nullptr) throw gcnew ArgumentNullException("ex"); #endif try { m_vw->l->predict(*ex->m_example); auto prediction = predictionFactory->Create(m_vw, ex->m_example); // as this is not a ring-based example it is not free'd m_vw->l->finish_example(*m_vw, *ex->m_example); return prediction; } CATCHRETHROW } public ref struct ParseJsonState { VowpalWabbit^ vw; List^ examples; }; example& get_example_from_pool(void* v) { interior_ptr state = (interior_ptr)v; auto ex = (*state)->vw->GetOrCreateNativeExample(); (*state)->examples->Add(ex); return *ex->m_example; } List^ VowpalWabbit::ParseDecisionServiceJson(cli::array^ json, int offset, int length, bool copyJson, [Out] VowpalWabbitDecisionServiceInteractionHeader^% header) { #if _DEBUG if (json == nullptr) throw gcnew ArgumentNullException("json"); #endif try { header = gcnew VowpalWabbitDecisionServiceInteractionHeader(); ParseJsonState^ state = gcnew ParseJsonState(); state->vw = this; state->examples = gcnew List(); try { auto ex = GetOrCreateNativeExample(); state->examples->Add(ex); v_array examples = v_init(); example* native_example = ex->m_example; examples.push_back(native_example); interior_ptr state_ptr = &state; pin_ptr data = &json[0]; data += offset; DecisionServiceInteraction interaction; if (m_vw->audit) VW::read_line_decision_service_json(*m_vw, examples, reinterpret_cast(data), length, copyJson, get_example_from_pool, &state, &interaction); else VW::read_line_decision_service_json(*m_vw, examples, reinterpret_cast(data), length, copyJson, get_example_from_pool, &state, &interaction); // finalize example VW::setup_examples(*m_vw, examples); // delete native array of pointers, keep examples examples.delete_v(); header->EventId = gcnew String(interaction.eventId.c_str()); header->Actions = gcnew cli::array((int)interaction.actions.size()); int index = 0; for (auto a : interaction.actions) header->Actions[index++] = (int)a; header->Probabilities = gcnew cli::array((int)interaction.probabilities.size()); index = 0; for (auto p : interaction.probabilities) header->Probabilities[index++] = p; header->ProbabilityOfDrop = interaction.probabilityOfDrop; return state->examples; } catch (...) { // cleanup for each (auto ex in state->examples) delete ex; throw; } } CATCHRETHROW } List^ VowpalWabbit::ParseJson(String^ line) { #if _DEBUG if (line == nullptr) throw gcnew ArgumentNullException("line"); #endif auto bytes = System::Text::Encoding::UTF8->GetBytes(line); auto valueHandle = GCHandle::Alloc(bytes, GCHandleType::Pinned); try { ParseJsonState^ state = gcnew ParseJsonState(); state->vw = this; state->examples = gcnew List(); try { auto ex = GetOrCreateNativeExample(); state->examples->Add(ex); v_array examples = v_init(); example* native_example = ex->m_example; examples.push_back(native_example); interior_ptr state_ptr = &state; if (m_vw->audit) VW::read_line_json(*m_vw, examples, reinterpret_cast(valueHandle.AddrOfPinnedObject().ToPointer()), get_example_from_pool, &state); else VW::read_line_json(*m_vw, examples, reinterpret_cast(valueHandle.AddrOfPinnedObject().ToPointer()), get_example_from_pool, &state); // finalize example VW::setup_examples(*m_vw, examples); // remember the input string for debugging purposes ex->VowpalWabbitString = line; return state->examples; } catch (...) { // cleanup for each (auto ex in state->examples) delete ex; throw; } } CATCHRETHROW finally { valueHandle.Free(); } } VowpalWabbitExample^ VowpalWabbit::ParseLine(String^ line) { #if _DEBUG if (line == nullptr) throw gcnew ArgumentNullException("line"); #endif auto ex = GetOrCreateNativeExample(); auto bytes = System::Text::Encoding::UTF8->GetBytes(line); auto valueHandle = GCHandle::Alloc(bytes, GCHandleType::Pinned); try { try { VW::read_line(*m_vw, ex->m_example, reinterpret_cast(valueHandle.AddrOfPinnedObject().ToPointer())); // finalize example VW::setup_example(*m_vw, ex->m_example); // remember the input string for debugging purposes ex->VowpalWabbitString = line; return ex; } catch (...) { delete ex; throw; } } CATCHRETHROW finally { valueHandle.Free(); } } void VowpalWabbit::Learn(String^ line) { #if _DEBUG if (String::IsNullOrEmpty(line)) throw gcnew ArgumentException("lines must not be empty. For multi-line examples use Learn(IEnumerable) overload."); #endif VowpalWabbitExample^ example = nullptr; try { example = ParseLine(line); Learn(example); } finally { delete example; } } void VowpalWabbit::Predict(String^ line) { #if _DEBUG if (String::IsNullOrEmpty(line)) throw gcnew ArgumentException("lines must not be empty. For multi-line examples use Predict(IEnumerable) overload."); #endif VowpalWabbitExample^ example = nullptr; try { example = ParseLine(line); Predict(example); } finally { delete example; } } generic TPrediction VowpalWabbit::Learn(String^ line, IVowpalWabbitPredictionFactory^ predictionFactory) { #if _DEBUG if (String::IsNullOrEmpty(line)) throw gcnew ArgumentException("lines must not be empty. For multi-line examples use Learn(IEnumerable) overload."); #endif VowpalWabbitExample^ example = nullptr; try { example = ParseLine(line); return Learn(example, predictionFactory); } finally { delete example; } } generic T VowpalWabbit::Predict(String^ line, IVowpalWabbitPredictionFactory^ predictionFactory) { #if _DEBUG if (String::IsNullOrEmpty(line)) throw gcnew ArgumentException("lines must not be empty. For multi-line examples use Learn(IEnumerable) overload."); #endif VowpalWabbitExample^ example = nullptr; try { example = ParseLine(line); return Predict(example, predictionFactory); } finally { delete example; } } void VowpalWabbit::Learn(IEnumerable^ lines) { #if _DEBUG if (lines == nullptr) throw gcnew ArgumentNullException("lines"); #endif auto examples = gcnew List; try { for each (auto line in lines) { auto ex = ParseLine(line); examples->Add(ex); Learn(ex); } auto empty = GetOrCreateNativeExample(); examples->Add(empty); empty->MakeEmpty(this); Learn(empty); } finally { for each (auto ex in examples) { delete ex; } } } void VowpalWabbit::Predict(IEnumerable^ lines) { #if _DEBUG if (lines == nullptr) throw gcnew ArgumentNullException("lines"); #endif auto examples = gcnew List; try { for each (auto line in lines) { auto ex = ParseLine(line); examples->Add(ex); Predict(ex); } auto empty = GetOrCreateNativeExample(); examples->Add(empty); empty->MakeEmpty(this); Predict(empty); } finally { for each (auto ex in examples) { delete ex; } } } generic T VowpalWabbit::Learn(IEnumerable^ lines, IVowpalWabbitPredictionFactory^ predictionFactory) { #if _DEBUG if (lines == nullptr) throw gcnew ArgumentNullException("lines"); #endif auto examples = gcnew List; try { for each (auto line in lines) { auto ex = ParseLine(line); examples->Add(ex); Learn(ex); } auto empty = GetOrCreateNativeExample(); examples->Add(empty); empty->MakeEmpty(this); Learn(empty); return examples[0]->GetPrediction(this, predictionFactory); } finally { for each (auto ex in examples) { delete ex; } } } generic T VowpalWabbit::Predict(IEnumerable^ lines, IVowpalWabbitPredictionFactory^ predictionFactory) { #if _DEBUG if (lines == nullptr) throw gcnew ArgumentNullException("lines"); #endif auto examples = gcnew List; try { for each (auto line in lines) { auto ex = ParseLine(line); examples->Add(ex); Predict(ex); } auto empty = GetOrCreateNativeExample(); examples->Add(empty); empty->MakeEmpty(this); Predict(empty); return examples[0]->GetPrediction(this, predictionFactory); } finally { for each (auto ex in examples) { delete ex; } } } void VowpalWabbit::EndOfPass() { try { m_vw->l->end_pass(); sync_stats(*m_vw); } CATCHRETHROW } /// /// Hashes the given value . /// /// String to be hashed. /// Hash offset. /// The resulting hash code. //template uint64_t hashall(String^ s, int offset, int count, uint64_t u) { // get raw bytes from string auto keys = gcnew cli::array(Encoding::UTF8->GetMaxByteCount(count)); int length = Encoding::UTF8->GetBytes(s, offset, count, keys, 0); // TOOD: benchmark and verify correctness //if (replaceSpace) //{ // for (int j = 0; j < length;) // { // var k = keys[j]; // if (k == ' ') // { // keys[j] = '_'; // } // j++; // // take care of UTF-8 multi-byte characters // while (k & 0xC == 0xC) // { // j++; // k <<= 1; // } // } //} uint32_t h1 = (uint32_t)u; uint32_t k1 = 0; const uint32_t c1 = 0xcc9e2d51; const uint32_t c2 = 0x1b873593; int i = 0; while (i <= length - 4) { // convert byte array to integer k1 = (uint32_t)(keys[i] | keys[i + 1] << 8 | keys[i + 2] << 16 | keys[i + 3] << 24); k1 *= c1; k1 = ROTL32(k1, 15); k1 *= c2; h1 ^= k1; h1 = ROTL32(h1, 13); h1 = h1 * 5 + 0xe6546b64; i += 4; } k1 = 0; int tail = length - length % 4; switch (length & 3) { case 3: k1 ^= (uint32_t)(keys[tail + 2] << 16); case 2: k1 ^= (uint32_t)(keys[tail + 1] << 8); case 1: k1 ^= (uint32_t)(keys[tail]); k1 *= c1; k1 = ROTL32(k1, 15); k1 *= c2; h1 ^= k1; break; } // finalization h1 ^= (uint32_t)length; return MURMUR_HASH_3::fmix(h1); } uint64_t hashall(String^ s, uint64_t u) { return hashall(s, 0, s->Length, u); } /// /// Hashes the given value . /// /// String to be hashed. /// Hash offset. /// The resulting hash code. size_t hashstring(String^ s, size_t u) { int offset = 0; int end = s->Length; if (end == 0) return u; //trim leading whitespace but not UTF-8 for (; offset < s->Length && s[offset] <= 0x20; offset++); for (; end >= offset && s[end - 1] <= 0x20; end--); int sInt = 0; for (int i = offset; i < end; i++) { auto c = s[i]; if (c >= '0' && c <= '9') sInt = 10 * sInt + (c - '0'); else return hashall(s, offset, end - offset, u); } return sInt + u; } Func^ VowpalWabbit::GetHasher() { //feature manipulation string hash_function("strings"); if (m_vw->vm.count("hash")) { hash_function = m_vw->vm["hash"].as(); } if (hash_function == "strings") { return gcnew Func(&hashstring); } else if (hash_function == "all") { return gcnew Func(&hashall); } else { THROW("Unsupported hash function: " << hash_function); } } VowpalWabbit^ VowpalWabbit::Native::get() { return this; } VowpalWabbitExample^ VowpalWabbit::GetOrCreateNativeExample() { auto ex = m_examples->Remove(); if (ex == nullptr) { try { auto ex = VW::alloc_examples(0, 1); m_vw->p->lp.default_label(&ex->l); return gcnew VowpalWabbitExample(this, ex); } CATCHRETHROW } try { VW::empty_example(*m_vw, *ex->m_example); m_vw->p->lp.default_label(&ex->m_example->l); return ex; } CATCHRETHROW } void VowpalWabbit::ReturnExampleToPool(VowpalWabbitExample^ ex) { #if _DEBUG if (m_vw == nullptr) throw gcnew ObjectDisposedException("VowpalWabbitExample was not properly disposed as the owner is already disposed"); #endif if (ex == nullptr) throw gcnew ArgumentNullException("ex"); // make sure we're not a ring based example assert(!VW::is_ring_example(*m_vw, ex->m_example)); // the bag might have reached it's limit if (m_examples != nullptr) { if (!m_examples->TryAdd(ex)) DisposeExample(ex); } #if _DEBUG else // this should not happen as m_vw is already set to null throw gcnew ObjectDisposedException("VowpalWabbitExample was disposed after the owner is disposed"); #endif } cli::array^>^ VowpalWabbit::GetTopicAllocation(int top) { uint64_t length = (uint64_t)1 << m_vw->num_bits; // using jagged array to enable LINQ auto K = (int)m_vw->lda; auto allocation = gcnew cli::array^>(K); // TODO: better way of peaking into lda? auto lda_rho = m_vw->vm["lda_rho"].as(); std::vector top_weights; // over topics for (int topic = 0; topic < K; topic++) { get_top_weights(m_vw, top, topic, top_weights); auto clr_weights = gcnew List(top); allocation[topic] = clr_weights; for (auto& pair : top_weights) clr_weights->Add(gcnew VowpalWabbitFeature(this, pair.x, pair.weight_index)); } return allocation; } template cli::array^>^ VowpalWabbit::FillTopicAllocation(T& weights) { uint64_t length = (uint64_t)1 << m_vw->num_bits; // using jagged array to enable LINQ auto K = (int)m_vw->lda; auto allocation = gcnew cli::array^>(K); for (int k = 0; k < K; k++) allocation[k] = gcnew cli::array((int)length); // TODO: better way of peaking into lda? auto lda_rho = m_vw->vm["lda_rho"].as(); for (auto iter = weights.begin(); iter != weights.end(); ++iter) { // over topics weight* wp = &(*iter); for (uint64_t k = 0; k < K; k++) allocation[(int)k][(int)iter.index()] = wp[k] + lda_rho; } return allocation; } cli::array^>^ VowpalWabbit::GetTopicAllocation() { // over weights if (m_vw->weights.sparse) return FillTopicAllocation(m_vw->weights.sparse_weights); else return FillTopicAllocation(m_vw->weights.dense_weights); } } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vowpalwabbit.h000066400000000000000000000240711323713302100213320ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #pragma once #include "vw_clr.h" #include "vw_base.h" #include "vw_model.h" #include "vw_prediction.h" #include "vw_interface.h" namespace VW { ref class VowpalWabbitExampleBuilder; ref struct VowpalWabbitFeature; /// /// Simple string example based wrapper for vowpal wabbit. /// /// If possible use VowpalWabbit{T} types as this wrapper suffers from marshalling performance wise. public ref class VowpalWabbit : VowpalWabbitBase, IVowpalWabbitExamplePool { private: /// /// Select the right hash method based on args. /// Func^ GetHasher(); /// /// The selected hasher method. /// /// /// Avoiding if-else for hash function selection. Delegates outperform function pointers according to http://stackoverflow.com/questions/13443250/performance-of-c-cli-function-pointers-versus-net-delegates /// initonly Func^ m_hasher; template cli::array^>^ FillTopicAllocation(T& weights); public: /// /// Initializes a new instance. /// /// The settings. VowpalWabbit(VowpalWabbitSettings^ settings); /// /// Initializes a new instance. /// /// Command line arguments. VowpalWabbit(String^ args); /// /// Run multi-passe training. /// void RunMultiPass(); /// /// Gets Collected performance statistics. /// property VowpalWabbitPerformanceStatistics^ PerformanceStatistics { VowpalWabbitPerformanceStatistics^ get(); } /// /// Parses using the C++ parser. /// /// /// Returns a ready to be used for or . /// VowpalWabbitExample^ ParseLine(String^ line); /// /// Parses using the C++ parser. /// TODO: this should return VowpalWabbitExampleCollection, but that would require moving VowpalWaabitExampleCollection to C++/CLI /// /// /// Returns a ready to be used for or . /// List^ ParseJson(String^ line); /// /// Parses using the C++ parser and supports the extra wrapping introduced by Decision Service. /// TODO: this should return VowpalWabbitExampleCollection, but that would require moving VowpalWaabitExampleCollection to C++/CLI /// TODO: the header should be passed along with the List of VowpalWabbit examples, but that requires additional care wrt disposing items. /// /// This needs to be null-terminated string. /// If true the json array is copied prior to destructive parsing /// /// Returns a ready to be used for or . /// List^ VowpalWabbit::ParseDecisionServiceJson(cli::array^ json, int offset, int length, bool copyJson, [Out] VowpalWabbitDecisionServiceInteractionHeader^% header); /// /// Hashes the given namespace . /// /// String to be hashed. /// The resulting hash code. /// The hash code depends on the vowpal wabbit instance as different has functions can be configured. uint64_t HashSpaceNative(String^ s); /// /// Hashes the given namespace . /// /// String to be hashed. /// The resulting hash code. /// The hash code depends on the vowpal wabbit instance as different has functions can be configured. uint64_t HashSpace(String^ s); /// /// Hash the given feature . /// /// String to be hashed. /// Hash offset. /// The resulting hash code. /// The hash code depends on the vowpal wabbit instance as different has functions can be configured. uint64_t HashFeatureNative(String^ s, size_t u); /// /// Hash the given feature . /// /// String to be hashed. /// Hash offset. /// The resulting hash code. /// The hash code depends on the vowpal wabbit instance as different has functions can be configured. uint64_t HashFeature(String^ s, size_t u); /// /// Return full topic allocation [topic, feature]. /// cli::array^>^ GetTopicAllocation(); /// /// Return the topic weights. /// cli::array^>^ GetTopicAllocation(int top); /// /// The associated instance learns from this example and returns the prediction result for this example. /// /// The prediction result. /// The prediction result type. generic T Learn(VowpalWabbitExample^ example, IVowpalWabbitPredictionFactory^ predictionFactory); /// /// Predicts for the given example. /// /// The prediction type. /// Example to predict for. /// The prediction factory to be used. See . /// The prediction for the given . generic T Predict(VowpalWabbitExample^ example, IVowpalWabbitPredictionFactory^ predictionFactory); /// /// Learns from the given example. /// /// Example to learn from. void Learn(VowpalWabbitExample^ example); /// /// Predicts for the given example. /// /// Example to predict for. void Predict(VowpalWabbitExample^ example); /// /// Learns from string data. /// /// Data in vw string format. void Learn(String^ line); /// /// Predicts for string data. /// /// Data in vw string format. void Predict(String^ line); /// /// Learns from string data. /// /// The prediction type. /// Data in vw string format. /// The prediction factory to be used. See . /// The prediction for the given . generic T Learn(String^ line, IVowpalWabbitPredictionFactory^ predictionFactory); /// /// Predicts for string data. /// /// The prediction type. /// Data in vw string format. /// The prediction factory to be used. See . /// The prediction for the given . generic T Predict(String^ line, IVowpalWabbitPredictionFactory^ predictionFactory); /// /// Learns from multi-line examples. /// /// Data in vw string format. void Learn(IEnumerable^ lines); /// /// Predicts for multi-line examples. /// /// Data in vw string format. void Predict(IEnumerable^ lines); /// /// Learns from multi-line examples. /// /// The prediction type. /// Data in vw string format. /// The prediction factory to be used. See . /// The prediction for the given . generic T Learn(IEnumerable^ lines, IVowpalWabbitPredictionFactory^ predictionFactory); /// /// Predicts for the given lines. /// /// The prediction type. /// Data in vw string format. /// The prediction factory to be used. See . /// The prediction for the given . generic T Predict(IEnumerable^ lines, IVowpalWabbitPredictionFactory^ predictionFactory); /// /// Signals the end of a pass. /// void EndOfPass(); /// /// Invokes the driver. /// void Driver(); virtual property VowpalWabbit^ Native { virtual VowpalWabbit^ get() sealed; } /// /// Gets or creates a native example from a CLR maintained, but natively allocated pool. /// /// A ready to use cleared native example data structure. virtual VowpalWabbitExample^ GetOrCreateNativeExample() sealed; /// /// Puts a native example data structure back into the pool. /// /// The example to be returned. virtual void ReturnExampleToPool(VowpalWabbitExample^ example) sealed; }; } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vw_arguments.h000066400000000000000000000053631323713302100213550ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #pragma once #include #include "vw.h" #include using namespace std; using namespace System; using namespace System::Text; using namespace System::Collections::Generic; namespace VW { /// /// command line arguments extracted from native C++. /// public ref class VowpalWabbitArguments { private: initonly String^ m_data; initonly String^ m_finalRegressor; const bool m_testonly; const int m_passes; List^ m_regressors; String^ m_commandLine; int m_numberOfActions; float m_learning_rate; float m_power_t; internal: VowpalWabbitArguments(vw* vw) : m_data(gcnew String(vw->data_filename.c_str())), m_finalRegressor(gcnew String(vw->final_regressor_name.c_str())), m_testonly(!vw->training), m_passes((int)vw->numpasses) { po::variables_map& vm = vw->vm; if (vm.count("initial_regressor") || vm.count("i")) { m_regressors = gcnew List; vector regs = vm["initial_regressor"].as< vector >(); for (auto& r : regs) m_regressors->Add(gcnew String(r.c_str())); } StringBuilder^ sb = gcnew StringBuilder(); for (auto& s : vw->args) sb->AppendFormat("{0} ", gcnew String(s.c_str())); m_commandLine = sb->ToString()->TrimEnd(); if (vw->vm.count("cb")) m_numberOfActions = (int)vw->vm["cb"].as(); m_learning_rate = vw->eta; m_power_t = vw->power_t; } public: /// /// The input data file. /// property String^ Data { String^ get() { return m_data; } } /// /// True if "-t" for test only mode supplied as part of arguments. /// property bool TestOnly { bool get() { return m_testonly; } } /// /// Number of passes. /// property int NumPasses { int get() { return m_passes; } } /// /// The output model filename. /// property String^ FinalRegressor { String^ get() { return m_finalRegressor; } } /// ///The list of input model filenames. /// property List^ InitialRegressors { List^ get() { return m_regressors; } } property String^ CommandLine { String^ get() { return m_commandLine; } } property int ContextualBanditNumberOfActions { int get() { return m_numberOfActions; } } property float LearningRate { float get() { return m_learning_rate; } } property float PowerT { float get() { return m_power_t; } } }; } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vw_base.cpp000066400000000000000000000156361323713302100206210ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #include "vw_clr.h" #include "vw_base.h" #include "vw_model.h" #include "vw_prediction.h" #include "vw_example.h" #include "clr_io.h" #include "clr_io_memory.h" #include "vw_exception.h" #include "parse_args.h" #include "parse_regressor.h" using namespace System; using namespace System::Collections::Generic; using namespace System::Text; namespace VW { void trace_listener_cli(void* context, const std::string& message) { auto listener = (Action^)GCHandle::FromIntPtr(IntPtr(context)).Target; auto str = gcnew String(message.c_str()); listener(str->TrimEnd()); } VowpalWabbitBase::VowpalWabbitBase(VowpalWabbitSettings^ settings) : m_examples(nullptr), m_vw(nullptr), m_model(nullptr), m_settings(settings != nullptr ? settings : gcnew VowpalWabbitSettings), m_instanceCount(0) { if (m_settings->EnableThreadSafeExamplePooling) m_examples = Bag::CreateLockFree(m_settings->MaxExamples); else m_examples = Bag::Create(m_settings->MaxExamples); try { try { std::string string; if (settings->Arguments != nullptr) string = msclr::interop::marshal_as(settings->Arguments); trace_message_t trace_listener = nullptr; void* trace_context = nullptr; if (settings->TraceListener != nullptr) { m_traceListener = GCHandle::Alloc(settings->TraceListener); trace_context = GCHandle::ToIntPtr(m_traceListener).ToPointer(); trace_listener = trace_listener_cli; } if (settings->Model != nullptr) { m_model = settings->Model; if (!settings->Verbose && !settings->Arguments->Contains("--quiet") && !m_model->Arguments->CommandLine->Contains("--quiet")) string.append(" --quiet"); m_vw = VW::seed_vw_model(m_model->m_vw, string, trace_listener, trace_context); m_model->IncrementReference(); } else { if (settings->ModelStream == nullptr) { if (!settings->Verbose && !settings->Arguments->Contains("--quiet")) string.append(" --quiet"); m_vw = VW::initialize(string, nullptr, false, trace_listener, trace_context); } else { clr_io_buf model(settings->ModelStream); if (!settings->Arguments->Contains("--no_stdin")) string += " --no_stdin"; m_vw = VW::initialize(string, &model, false, trace_listener, trace_context); delete settings->ModelStream; settings->ModelStream = nullptr; } } } catch (...) { // memory leak, but better than crashing m_vw = nullptr; throw; } } CATCHRETHROW } VowpalWabbitBase::~VowpalWabbitBase() { this->!VowpalWabbitBase(); if (m_traceListener.IsAllocated) m_traceListener.Free(); } VowpalWabbitBase::!VowpalWabbitBase() { if (m_instanceCount <= 0) { this->InternalDispose(); } } void VowpalWabbitBase::IncrementReference() { // thread-safe increase of model reference counter System::Threading::Interlocked::Increment(m_instanceCount); } void VowpalWabbitBase::DecrementReference() { // thread-safe decrease of model reference counter if (System::Threading::Interlocked::Decrement(m_instanceCount) <= 0) { this->InternalDispose(); } } void VowpalWabbitBase::DisposeExample(VowpalWabbitExample^ ex) { VW::dealloc_example(m_vw->p->lp.delete_label, *ex->m_example); ::free_it(ex->m_example); // cleanup pointers in example chain auto inner = ex; while ((inner = inner->InnerExample) != nullptr) { inner->m_owner = nullptr; inner->m_example = nullptr; } ex->m_example = nullptr; // avoid that this example is returned again ex->m_owner = nullptr; } void VowpalWabbitBase::InternalDispose() { if (m_vw != nullptr) { // de-allocate example pools that are managed for each even shared instances if (m_examples != nullptr) { for each (auto ex in m_examples->RemoveAll()) DisposeExample(ex); m_examples = nullptr; } if (m_model != nullptr) { // this object doesn't own the VW instance m_model->DecrementReference(); m_model = nullptr; } } try { if (m_vw != nullptr) { reset_source(*m_vw, m_vw->num_bits); release_parser_datastructures(*m_vw); // make sure don't try to free m_vw twice in case VW::finish throws. vw* vw_tmp = m_vw; m_vw = nullptr; VW::finish(*vw_tmp); } // don't add code here as in the case of VW::finish throws an exception it won't be called } CATCHRETHROW } VowpalWabbitSettings^ VowpalWabbitBase::Settings::get() { return m_settings; } VowpalWabbitArguments^ VowpalWabbitBase::Arguments::get() { if (m_arguments == nullptr) { m_arguments = gcnew VowpalWabbitArguments(m_vw); } return m_arguments; } void VowpalWabbitBase::Reload([System::Runtime::InteropServices::Optional] String^ args) { if (m_settings->ParallelOptions != nullptr) { throw gcnew NotSupportedException("Cannot reload model if AllRecude is enabled."); } clr_io_memory_buf mem_buf; if (args == nullptr) args = String::Empty; auto stringArgs = msclr::interop::marshal_as(args); try { reset_source(*m_vw, m_vw->num_bits); VW::save_predictor(*m_vw, mem_buf); mem_buf.flush(); release_parser_datastructures(*m_vw); // make sure don't try to free m_vw twice in case VW::finish throws. vw* vw_tmp = m_vw; m_vw = nullptr; VW::finish(*vw_tmp); // reload from model // seek to beginning mem_buf.reset_file(0); m_vw = VW::initialize(stringArgs.c_str(), &mem_buf); } CATCHRETHROW } String^ VowpalWabbitBase::AreFeaturesCompatible(VowpalWabbitBase^ other) { auto diff = VW::are_features_compatible(*m_vw, *other->m_vw); return diff == nullptr ? nullptr : gcnew String(diff); } String^ VowpalWabbitBase::ID::get() { return gcnew String(m_vw->id.c_str()); } void VowpalWabbitBase::ID::set(String^ value) { m_vw->id = msclr::interop::marshal_as(value); } void VowpalWabbitBase::SaveModel() { string name = m_vw->final_regressor_name; if (name.empty()) { return; } // this results in extra marshaling but should be fine here this->SaveModel(gcnew String(name.c_str())); } void VowpalWabbitBase::SaveModel(String^ filename) { if (String::IsNullOrEmpty(filename)) throw gcnew ArgumentException("Filename must not be null or empty"); String^ directoryName = System::IO::Path::GetDirectoryName(filename); if (!String::IsNullOrEmpty(directoryName)) { System::IO::Directory::CreateDirectory(directoryName); } auto name = msclr::interop::marshal_as(filename); try { VW::save_predictor(*m_vw, name); } CATCHRETHROW } void VowpalWabbitBase::SaveModel(Stream^ stream) { if (stream == nullptr) throw gcnew ArgumentException("stream"); try { VW::clr_io_buf buf(stream); VW::save_predictor(*m_vw, buf); } CATCHRETHROW } } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vw_base.h000066400000000000000000000101261323713302100202530ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #pragma once #include "vw_clr.h" #include #include "vw_interface.h" #include "vw_arguments.h" using namespace std; using namespace System::Collections::Generic; namespace VW { ref class VowpalWabbitPrediction; ref class VowpalWabbitModel; /// /// A base wrapper around vowpal wabbit machine learning instance. /// /// /// Since the model class must delay diposal of until all referencing /// VowpalWabbit instances are disposed, the base class does not dispose . /// public ref class VowpalWabbitBase abstract { private: /// /// The settings used for this instance. /// initonly VowpalWabbitSettings^ m_settings; /// /// Handle to trace listener delegate, required to keep safe from garbage collection. /// GCHandle m_traceListener; /// /// An optional shared model. /// VowpalWabbitModel^ m_model; /// /// Extracted command line arguments. /// VowpalWabbitArguments^ m_arguments; /// /// Reference count to native data structure. /// System::Int32 m_instanceCount; internal: /// /// The native vowpal wabbit data structure. /// vw* m_vw; /// /// Thread-safe increment of reference count. /// void IncrementReference(); /// /// Thread-safe decrement of reference count. /// void DecrementReference(); protected: /// /// True if all nativedata structures are disposed. /// bool m_isDisposed; /// /// Example pool. Kept in base to simplify deallocation. /// IBag^ m_examples; /// /// Initializes a new instance. /// /// Command line arguments. VowpalWabbitBase(VowpalWabbitSettings^ settings); /// /// Cleanup. /// !VowpalWabbitBase(); /// /// Internal dipose using reference counting to delay disposal of shared native data structures. /// void InternalDispose(); void DisposeExample(VowpalWabbitExample^ ex); public: /// /// Cleanup. /// virtual ~VowpalWabbitBase(); /// /// The settings used for this instance. /// property VowpalWabbitSettings^ Settings { VowpalWabbitSettings^ get(); } /// /// Extracted command line arguments. /// property VowpalWabbitArguments^ Arguments { VowpalWabbitArguments^ get(); } /// /// The read/writable model id. /// property String^ ID { String^ get(); void set(String^ id); } /// /// Performs the following steps to reset the learning state: /// /// - Save model to in-memory buffer /// - Dispose existing instance /// - Initialize new instance from in-memory buffer /// void Reload([System::Runtime::InteropServices::Optional] String^ args); /// /// Compares features created by current instance are compatible to features created by . /// /// /// Null if compatible, otherwise the difference /// String^ AreFeaturesCompatible(VowpalWabbitBase^ other); /// /// Persist model to file specified by -i. /// void SaveModel(); /// /// Persist model to . /// /// The destination filename for the model. void SaveModel(String^ filename); /// /// Persist model to . /// /// The destination stream for the model. /// The stream is not closed to support embedded schemes. void SaveModel(Stream^ stream); }; } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vw_builder.cpp000066400000000000000000000062471323713302100213330ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #include "vw_builder.h" #include "parser.h" namespace VW { VowpalWabbitExampleBuilder::VowpalWabbitExampleBuilder(IVowpalWabbitExamplePool^ vw) : m_vw(vw), m_example(nullptr) { if (vw == nullptr) throw gcnew ArgumentNullException("vw"); m_example = vw->GetOrCreateNativeExample(); } VowpalWabbitExampleBuilder::~VowpalWabbitExampleBuilder() { this->!VowpalWabbitExampleBuilder(); } VowpalWabbitExampleBuilder::!VowpalWabbitExampleBuilder() { if (m_example != nullptr) { // in case CreateExample is not getting called delete m_example; m_example = nullptr; } } VowpalWabbitExample^ VowpalWabbitExampleBuilder::CreateExample() { if (m_example == nullptr) return nullptr; try { // finalize example VW::setup_example(*m_vw->Native->m_vw, m_example->m_example); } CATCHRETHROW // hand memory management off to VowpalWabbitExample auto ret = m_example; m_example = nullptr; return ret; } void VowpalWabbitExampleBuilder::ApplyLabel(ILabel^ label) { if (label == nullptr) return; label->UpdateExample(m_vw->Native->m_vw, m_example->m_example); } VowpalWabbitNamespaceBuilder^ VowpalWabbitExampleBuilder::AddNamespace(Char featureGroup) { return AddNamespace((Byte)featureGroup); } VowpalWabbitNamespaceBuilder^ VowpalWabbitExampleBuilder::AddNamespace(Byte featureGroup) { uint32_t index = featureGroup; example* ex = m_example->m_example; return gcnew VowpalWabbitNamespaceBuilder(ex->feature_space + index, featureGroup, m_example->m_example); } VowpalWabbitNamespaceBuilder::VowpalWabbitNamespaceBuilder(features* features, unsigned char index, example* example) : m_features(features), m_index(index), m_example(example) { } VowpalWabbitNamespaceBuilder::~VowpalWabbitNamespaceBuilder() { this->!VowpalWabbitNamespaceBuilder(); } VowpalWabbitNamespaceBuilder::!VowpalWabbitNamespaceBuilder() { if (m_features->size() > 0) { unsigned char temp = m_index; // avoid duplicate insertion // can't check at the beginning, because multiple builders can be open // at the same time for (unsigned char ns : m_example->indices) if (ns == temp) return; m_example->indices.push_back(temp); } } void VowpalWabbitNamespaceBuilder::AddFeaturesUnchecked(uint64_t weight_index_base, float* begin, float* end) { for (; begin != end; begin++) { float x = *begin; if (x != 0) { m_features->values.push_back_unchecked(x); m_features->indicies.push_back_unchecked(weight_index_base); } weight_index_base++; } } void VowpalWabbitNamespaceBuilder::AddFeature(uint64_t weight_index, float x) { // filter out 0-values if (x == 0) return; m_features->push_back(x, weight_index); } void VowpalWabbitNamespaceBuilder::PreAllocate(int size) { m_features->values.resize(m_features->values.end() - m_features->values.begin() + size); m_features->indicies.resize(m_features->indicies.end() - m_features->indicies.begin() + size); } size_t VowpalWabbitNamespaceBuilder::FeatureCount::get() { return m_features->size(); } } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vw_builder.h000066400000000000000000000072011323713302100207670ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #pragma once #include "vw_clr.h" #include "vw_base.h" #include "vw_example.h" #include "vowpalwabbit.h" #include "vw_label.h" namespace VW { using namespace VW::Labels; /// /// Helper class to ease construction of native vowpal wabbit namespace data structure. /// public ref class VowpalWabbitNamespaceBuilder sealed { private: /// /// Features. /// features* m_features; /// /// The namespace index. /// unsigned char m_index; /// /// The native example. /// example* m_example; // float(*m_sum_of_squares)(float*, float*); !VowpalWabbitNamespaceBuilder(); internal: /// /// Initializes a new instance. /// /// Pointer into features owned by . /// The namespace index. /// The native example to build up. VowpalWabbitNamespaceBuilder(features* features, unsigned char index, example* example); public: ~VowpalWabbitNamespaceBuilder(); /// /// Add feature entry. /// /// The weight index. /// The value. void AddFeature(uint64_t weight_index, float x); /// /// Adds a dense array to the example. /// /// The base weight index. Each element is then placed relative to this index. /// The start pointer of the float array. /// The end pointer of the float array. void AddFeaturesUnchecked(uint64_t weight_index_base, float* begin, float* end); /// /// Pre-allocate features of . /// /// The number of features to pre-allocate. void PreAllocate(int size); property size_t FeatureCount { size_t get(); } }; /// /// Helper class to ease construction of native vowpal wabbit example data structure. /// public ref class VowpalWabbitExampleBuilder sealed { private: IVowpalWabbitExamplePool^ m_vw; /// /// The produced CLR example data structure. /// VowpalWabbitExample^ m_example; protected: /// /// Cleanup. /// !VowpalWabbitExampleBuilder(); public: /// /// Initializes a new instance. /// /// The parent vowpal wabbit instance. VowpalWabbitExampleBuilder(IVowpalWabbitExamplePool^ vw); /// /// Cleanup. /// ~VowpalWabbitExampleBuilder(); /// /// Creates the managed example representation. /// /// Creates the managed example. VowpalWabbitExample^ CreateExample(); /// /// Sets the label for the resulting example. /// void ApplyLabel(ILabel^ label); /// /// Creates and adds a new namespace to this example. /// VowpalWabbitNamespaceBuilder^ AddNamespace(Byte featureGroup); /// /// Creates and adds a new namespace to this example. /// /// The feature group of the new namespace. /// Casts to System::Byte. VowpalWabbitNamespaceBuilder^ AddNamespace(Char featureGroup); }; } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vw_cbutil.cpp000066400000000000000000000010221323713302100211510ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #include "vw_cbutil.h" #include "cb_algs.h" namespace VW { float VowpalWabbitContextualBanditUtil::GetUnbiasedCost(uint32_t actionObservered, uint32_t actionTaken, float cost, float probability) { CB::cb_class observation = { cost, actionObservered, probability }; return CB_ALGS::get_unbiased_cost(&observation, actionTaken); } } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vw_cbutil.h000066400000000000000000000006661323713302100206330ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #pragma once #include "vw_clr.h" namespace VW { public ref class VowpalWabbitContextualBanditUtil abstract sealed { public: static float GetUnbiasedCost(uint32_t actionObservered, uint32_t actionTaken, float cost, float probability); }; } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vw_clr.h000066400000000000000000000102631323713302100201230ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #pragma once #ifdef _MSC_VER #define _CRT_SECURE_NO_WARNINGS #endif #include "vw.h" #include "vw_settings.h" #include using namespace System; using namespace System::Runtime::InteropServices; namespace VW { /// /// Collected performance statistics. /// public ref class VowpalWabbitPerformanceStatistics { public: /// /// The total number of features seen since instance creation. /// property uint64_t TotalNumberOfFeatures; /// /// The weighted sum of examples. /// property double WeightedExampleSum; /// /// The total number of examples per pass. /// property uint64_t NumberOfExamplesPerPass; /// /// The weighted sum of labels. /// property double WeightedLabelSum; /// /// The average loss since instance creation. /// property double AverageLoss; /// /// The best constant since instance creation. /// property double BestConstant; /// /// The best constant loss since instance creation. /// property double BestConstantLoss; }; /// /// A managed wrapper for native vowpal wabbit exceptions. /// /// /// As the default managed exception wrapping any native exception doesn't even capture exception::what() /// this wrapper was created. /// [Serializable] public ref class VowpalWabbitException : Exception { private: /// /// The source filename in which the wrapped exception occurred. /// initonly String^ m_filename; /// /// The line number in which the wrapped exception occurred. /// initonly Int32 m_lineNumber; public: /// /// Initializes a new instance of . /// /// The native vowpal wabbit exception VowpalWabbitException(const vw_exception& ex); /// /// Gets the source filename in which the wrapped exception occurred. /// property String^ Filename { String^ get(); } /// /// Gets the line number in which the wrapped exception occurred. /// property Int32 LineNumber { Int32 get(); } }; /// /// A managed wrapper for native vowpal wabbit exceptions. /// /// /// As the default managed exception wrapping any native exception doesn't even capture exception::what() /// this wrapper was created. /// [Serializable] public ref class VowpalWabbitArgumentDisagreementException : VowpalWabbitException { public: /// /// Initializes a new instance of . /// /// The native vowpal wabbit exception VowpalWabbitArgumentDisagreementException(const vw_argument_disagreement_exception& ex); }; #ifdef _DEBUG [System::ComponentModel::Browsable(false)] [System::ComponentModel::EditorBrowsable(System::ComponentModel::EditorBrowsableState::Never)] public ref class VowpalWabbitLeakTest abstract sealed { public: static void Leak() { new float[123]; } static void NoLeak() { void* ptr = calloc(128, 2); ptr = realloc(ptr, 128 * 3); free(ptr); } }; #endif } #define CATCHRETHROW \ catch (VW::vw_exception const& ex) \ { throw gcnew VW::VowpalWabbitException(ex); } \ catch (std::exception const& ex) \ { throw gcnew System::Exception(gcnew System::String(ex.what())); } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vw_clr.vcxproj000066400000000000000000001166041323713302100213750ustar00rootroot00000000000000 DebugLeakCheck Win32 DebugLeakCheck x64 Debug Win32 Debug x64 Release Win32 Release x64 {85E55AE0-3784-4968-9271-C81AF560E1C1} v4.5.2 ManagedCProj vw_clr ..\..\vowpalwabbit\ ..\..\sdl\SDL-7.0-Recommended.ruleset true DynamicLibrary true v140 true Unicode DynamicLibrary true v140 true Unicode DynamicLibrary true v140 true Unicode DynamicLibrary true v140 true Unicode true DynamicLibrary false v140 true Unicode DynamicLibrary false v140 true Unicode 8.1 true ..\vw_key.snk true ..\vw_key.snk false VowpalWabbit.Core false ..\vw_key.snk $(Platform)\$(Configuration)\$(ProjectName)\ true VowpalWabbit.Core false ..\vw_key.snk $(Platform)\$(Configuration)\$(ProjectName)\ false ..\vw_key.snk false VowpalWabbit.Core ..\vw_key.snk $(Platform)\$(Configuration)\$(ProjectName)\ Level3 Disabled WIN32;_DEBUG;%(PreprocessorDefinitions) NotUsing $(SolutionDir)win32;$(ProjectDir)\..\..\vowpalwabbit;%(AdditionalIncludeDirectories);$(ProjectDir)\..\..\rapidjson\include true $(BoostLibDir);%(AdditionalLibraryDirectories) kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;ws2_32.lib;$(SolutionDir)$(PlatformShortName)\$(Configuration)\vw_dynamic.lib;%(AdditionalDependencies) Level3 Disabled WIN32;_DEBUG;%(PreprocessorDefinitions) NotUsing $(SolutionDir)win32;$(ProjectDir)\..\..\vowpalwabbit;%(AdditionalIncludeDirectories);$(ProjectDir)\..\..\rapidjson\include true $(BoostLibDir);%(AdditionalLibraryDirectories) kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;ws2_32.lib;$(SolutionDir)$(PlatformShortName)\$(Configuration)\vw_dynamic.lib;%(AdditionalDependencies) Level3 Disabled WIN32;_DEBUG;%(PreprocessorDefinitions) NotUsing $(SolutionDir)win32;$(ProjectDir)\..\..\vowpalwabbit;%(AdditionalIncludeDirectories);$(ProjectDir)\..\..\rapidjson\include true $(OutDir)$(TargetName).pdb false true false true kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;ws2_32.lib;%(AdditionalDependencies) true Level3 Disabled WIN32;_DEBUG;%(PreprocessorDefinitions) NotUsing $(SolutionDir)win32;$(ProjectDir)\..\..\vowpalwabbit;%(AdditionalIncludeDirectories);$(ProjectDir)\..\..\rapidjson\include true $(OutDir)$(TargetName).pdb false true false true kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;ws2_32.lib;%(AdditionalDependencies) Level3 WIN32;NDEBUG;_WINDOWS;%(PreprocessorDefinitions) NotUsing $(SolutionDir)win32;$(ProjectDir)\..\..\vowpalwabbit;%(AdditionalIncludeDirectories);$(ProjectDir)\..\..\rapidjson\include true $(BoostLibDir);%(AdditionalLibraryDirectories) kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;ws2_32.lib;$(SolutionDir)$(PlatformShortName)\$(Configuration)\vw_dynamic.lib;%(AdditionalDependencies) Level3 WIN32;NDEBUG;_WINDOWS;%(PreprocessorDefinitions) NotUsing $(SolutionDir)win32;$(ProjectDir)\..\..\vowpalwabbit;%(AdditionalIncludeDirectories);$(ProjectDir)\..\..\rapidjson\include true $(OutDir)$(TargetName).pdb true false true kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;ws2_32.lib;%(AdditionalDependencies) true false $(SolutionDir)\packages\Newtonsoft.Json.8.0.3\lib\net45\Newtonsoft.Json.dll {1e205806-7f80-47dd-a38d-fc08083f3593} {e621e022-c1f8-433f-905a-ab9a3de072b7} Designer This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. vowpal-wabbit-8.5.0.dfsg1/cs/cli/vw_clr.vcxproj.filters000066400000000000000000000073151323713302100230420ustar00rootroot00000000000000 {4FC737F1-C7A5-4376-A066-2A32D752A2FF} cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx {93995380-89BD-4b04-88EB-625FBE52EBFB} h;hh;hpp;hxx;hm;inl;inc;xsd {67DA6AB6-F800-4c08-8B7A-83BB121AAD01} rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx;tiff;tif;png;wav;mfcribbon-ms Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Resource Files vowpal-wabbit-8.5.0.dfsg1/cs/cli/vw_example.cpp000066400000000000000000000324341323713302100213350ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #include "vowpalwabbit.h" #include "vw_example.h" #include "vw_prediction.h" #include "gd.h" namespace VW { using namespace Labels; VowpalWabbitExample::VowpalWabbitExample(IVowpalWabbitExamplePool^ owner, example* example) : m_owner(owner), m_example(example), m_innerExample(nullptr) { } VowpalWabbitExample::VowpalWabbitExample(IVowpalWabbitExamplePool^ owner, VowpalWabbitExample^ example) : m_owner(owner), m_example(example->m_example), m_innerExample(example), m_string(example->m_string) { } VowpalWabbitExample::!VowpalWabbitExample() { if (m_owner != nullptr) m_owner->ReturnExampleToPool(this); } VowpalWabbitExample::~VowpalWabbitExample() { this->!VowpalWabbitExample(); } VowpalWabbitExample^ VowpalWabbitExample::InnerExample::get() { return m_innerExample; } IVowpalWabbitExamplePool^ VowpalWabbitExample::Owner::get() { return m_owner; } size_t VowpalWabbitExample::NumberOfFeatures::get() { return m_example->num_features; } generic T VowpalWabbitExample::GetPrediction(VowpalWabbit^ vw, IVowpalWabbitPredictionFactory^ factory) { #ifdef _DEBUG if (vw == nullptr) throw gcnew ArgumentNullException("vw"); #endif return factory->Create(vw->m_vw, m_example); } String^ VowpalWabbitExample::VowpalWabbitString::get() { return m_string; } void VowpalWabbitExample::VowpalWabbitString::set(String^ value) { m_string = value; } bool VowpalWabbitExample::IsNewLine::get() { return example_is_newline(*m_example) != 0; } ILabel^ VowpalWabbitExample::Label::get() { ILabel^ label; auto lp = m_owner->Native->m_vw->p->lp; if (!memcmp(&lp, &simple_label, sizeof(lp))) label = gcnew SimpleLabel(); else if (!memcmp(&lp, &CB::cb_label, sizeof(lp))) label = gcnew ContextualBanditLabel(); else if (!memcmp(&lp, &CB_EVAL::cb_eval, sizeof(lp))) label = gcnew SimpleLabel(); else if (!memcmp(&lp, &COST_SENSITIVE::cs_label, sizeof(lp))) label = gcnew SimpleLabel(); else return nullptr; // TODO: //else if (!memcmp(&lp, &MULTICLASS::multilabel, sizeof(lp))) // label = gcnew MulticlassLabel; //else if (!memcmp(&lp, &MC::multilabel, sizeof(lp))) label->ReadFromExample(this->m_example); return label; } void VowpalWabbitExample::Label::set(ILabel^ label) { if (label == nullptr) return; label->UpdateExample(m_owner->Native->m_vw, m_example); // we need to update the example weight as setup_example() can be called prior to this call. m_example->weight = m_owner->Native->m_vw->p->lp.get_weight(&m_example->l); } void VowpalWabbitExample::MakeEmpty(VowpalWabbit^ vw) { char empty = '\0'; VW::read_line(*vw->m_vw, m_example, &empty); VW::setup_example(*vw->m_vw, m_example); } void FormatIndices(example* a, System::Text::StringBuilder^ sb) { for (auto ns : a->indices) { if (ns == 0) sb->Append("NULL:0,"); else sb->AppendFormat("'{0}':{1},", gcnew System::Char(ns), (int)ns); } } System::String^ FormatIndices(example* a, example *b) { auto sb = gcnew System::Text::StringBuilder(); sb->AppendFormat("Namespace indicies differ: {0} vs {1}. this.indices: [", a->indices.size(), b->indices.size()); FormatIndices(a, sb); sb->Append("] other.indices: ["); FormatIndices(b, sb); sb->Append("]"); return sb->ToString(); } System::String^ FormatFeature(vw* vw, feature_value& f1, feature_index& i1) { uint64_t masked_weight_index1 = i1 & vw->weights.mask(); return System::String::Format( "weight_index = {0}/{1}, x = {2}", masked_weight_index1, i1, gcnew System::Single(f1)); } System::String^ FormatFeature(vw* vw, feature_value& f1, feature_index& i1, feature_value& f2, feature_index& i2) { return System::String::Format( "Feature differ: this({0}) vs other({1})", FormatFeature(vw, f1, i1), FormatFeature(vw, f2, i2)); } bool FloatEqual(float a, float b) { if ((abs(a) < 1e-20 && abs(b) < 1e-20) || (isinf(a) && isinf(b))) { return true; } return abs(a - b) / max(a, b) < 1e-6; } System::String^ FormatFeatures(vw* vw, features& arr) { auto sb = gcnew System::Text::StringBuilder(); for (size_t i = 0; i < arr.values.size(); i++) { sb->Append(FormatFeature(vw, arr.values[i], arr.indicies[i]))->Append(" "); } return sb->ToString(); } System::String^ CompareFeatures(vw* vw, features& fa, features& fb, unsigned char ns) { vector fa_missing; for (size_t ia = 0, ib = 0; ia < fa.values.size(); ia++) { auto masked_weight_index = fa.indicies[ia] & vw->weights.mask(); auto other_masked_weight_index = fb.indicies[ib] & vw->weights.mask(); /*System::Diagnostics::Debug::WriteLine(System::String::Format("{0} -> {1} vs {2} -> {3}", fa.indicies[ia], masked_weight_index, fb.indicies[ib], other_masked_weight_index ));*/ if (masked_weight_index == other_masked_weight_index && FloatEqual(fa.values[ia], fb.values[ib])) ib++; else { // fallback to search size_t ib_old = ib; bool found = false; for (ib = 0; ib < fb.values.size(); ib++) { auto other_masked_weight_index = fb.indicies[ib] & vw->weights.mask(); if (masked_weight_index == other_masked_weight_index) { if (!FloatEqual(fa.values[ia], fb.values[ib])) { return FormatFeature(vw, fa.values[ia], fa.indicies[ia], fb.values[ib], fb.indicies[ib]); } else { found = true; break; } } } if (!found) { fa_missing.push_back(ia); } ib = ib_old + 1; } } if (!fa_missing.empty()) { auto diff = gcnew System::Text::StringBuilder(); diff->AppendFormat("missing features in ns '{0}'/'{1}': ", ns, gcnew Char(ns)); for (size_t& ia : fa_missing) { diff->AppendFormat("this.weight_index = {0}, x = {1}, ", fa.indicies[ia] & vw->weights.mask(), fa.values[ia]); } return diff->ToString(); } return nullptr; } System::String^ VowpalWabbitExample::Diff(VowpalWabbit^ vw, VowpalWabbitExample^ other, IVowpalWabbitLabelComparator^ labelComparator) { auto a = this->m_example; auto b = other->m_example; if (a->indices.size() != b->indices.size()) { return FormatIndices(a, b); } for (auto i = a->indices.begin(), j = b->indices.begin(); i != a->indices.end(); i++) { if (*i == *j) j++; else { // fall back on search auto j_old = j; j = b->indices.begin(); bool found = false; for (; j != b->indices.end(); j++) { if (*i == *j) { found = true; break; } } if (!found) return FormatIndices(a, b); j = j_old + 1; } // compare features features& fa = a->feature_space[*i]; features& fb = b->feature_space[*i]; if (fa.size() != fb.size()) return System::String::Format("Feature length differ {0} vs {1}. this({2}) vs other({3})", fa.size(), fb.size(), FormatFeatures(vw->m_vw, fa), FormatFeatures(vw->m_vw, fb)); auto diff = CompareFeatures(vw->m_vw, fa, fb, *i); if (diff != nullptr) return diff; diff = CompareFeatures(vw->m_vw, fb, fa, *i); if (diff != nullptr) return diff; } if (labelComparator != nullptr) { // Compare the label auto diff = labelComparator->Diff(this, other); if (diff != nullptr) return diff; } return nullptr; } String^ VowpalWabbitSimpleLabelComparator::Diff(VowpalWabbitExample^ ex1, VowpalWabbitExample^ ex2) { auto s1 = ex1->m_example->l.simple; auto s2 = ex2->m_example->l.simple; if (!(FloatEqual(s1.initial, s2.initial) && FloatEqual(s1.label, s2.label) && FloatEqual(s1.weight, s2.weight))) { return System::String::Format("Label differ. label {0} vs {1}. initial {2} vs {3}. weight {4} vs {5}", s1.label, s2.label, s1.initial, s2.initial, s1.weight, s2.weight); } return nullptr; } String^ VowpalWabbitContextualBanditLabelComparator::Diff(VowpalWabbitExample^ ex1, VowpalWabbitExample^ ex2) { auto s1 = ex1->m_example->l.cb; auto s2 = ex2->m_example->l.cb; if (s1.costs.size() != s2.costs.size()) { return System::String::Format("Cost size differ: {0} vs {1}", s1.costs.size(), s2.costs.size()); } for (size_t i = 0; i < s1.costs.size(); i++) { auto c1 = s1.costs[i]; auto c2 = s2.costs[i]; if (c1.action != c2.action) { return System::String::Format("Action differ: {0} vs {1}", c1.action, c2.action); } if (c1.cost != c2.cost) { return System::String::Format("Cost differ: {0} vs {1}", c1.cost, c2.cost); } if (abs(c1.probability - c2.probability) / max(c1.probability, c2.probability) > 0.01) { return System::String::Format("Probability differ: {0} vs {1}", c1.probability, c2.probability); } } return nullptr; } System::Collections::IEnumerator^ VowpalWabbitExample::EnumerableGetEnumerator::get() { return GetEnumerator(); } IEnumerator^ VowpalWabbitExample::GetEnumerator() { return gcnew NamespaceEnumerator(this); } VowpalWabbitExample::NamespaceEnumerator::NamespaceEnumerator(VowpalWabbitExample^ example) : m_example(example) { Reset(); } VowpalWabbitExample::NamespaceEnumerator::~NamespaceEnumerator() { } bool VowpalWabbitExample::NamespaceEnumerator::MoveNext() { m_current++; return m_current < m_example->m_example->indices.end(); } void VowpalWabbitExample::NamespaceEnumerator::Reset() { // position before the beginning. m_current = m_example->m_example->indices.begin() - 1; } VowpalWabbitNamespace^ VowpalWabbitExample::NamespaceEnumerator::Current::get() { if (m_current < m_example->m_example->indices.begin() || m_current >= m_example->m_example->indices.end()) throw gcnew InvalidOperationException(); return gcnew VowpalWabbitNamespace(m_example, *m_current, &m_example->m_example->feature_space[*m_current]); } System::Object^ VowpalWabbitExample::NamespaceEnumerator::IEnumeratorCurrent::get() { return Current; } VowpalWabbitFeature::VowpalWabbitFeature(VowpalWabbitExample^ example, feature_value x, uint64_t weight_index) : m_example(example), m_vw(m_example->Owner->Native), m_x(x), m_weight_index(weight_index) { } VowpalWabbitFeature::VowpalWabbitFeature(VowpalWabbit^ vw, feature_value x, uint64_t weight_index) : m_vw(vw), m_x(x), m_weight_index(weight_index) { } float VowpalWabbitFeature::X::get() { return m_x; } uint64_t VowpalWabbitFeature::FeatureIndex::get() { return m_weight_index; } uint64_t VowpalWabbitFeature::WeightIndex::get() { if (m_example == nullptr) throw gcnew InvalidOperationException("VowpalWabbitFeature must be initialized with example"); vw* vw = m_example->Owner->Native->m_vw; return ((m_weight_index + m_example->m_example->ft_offset) >> vw->weights.stride_shift()) & vw->parse_mask; } float VowpalWabbitFeature::Weight::get() { if (m_example == nullptr) throw gcnew InvalidOperationException("VowpalWabbitFeature must be initialized with example"); vw* vw = m_example->Owner->Native->m_vw; uint64_t weightIndex = m_weight_index + m_example->m_example->ft_offset; return vw->weights[weightIndex]; } float VowpalWabbitFeature::AuditWeight::get() { vw* vw = m_vw->m_vw; return GD::trunc_weight(Weight, (float)vw->sd->gravity) * (float)vw->sd->contraction; } bool VowpalWabbitFeature::Equals(Object^ o) { VowpalWabbitFeature^ other = dynamic_cast(o); return other != nullptr && other->m_x == m_x && other->m_weight_index == m_weight_index; } int VowpalWabbitFeature::GetHashCode() { return (int)(m_x + m_weight_index); } VowpalWabbitNamespace::VowpalWabbitNamespace(VowpalWabbitExample^ example, namespace_index ns, features* features) : m_example(example), m_ns(ns), m_features(features) { } VowpalWabbitNamespace::~VowpalWabbitNamespace() { } namespace_index VowpalWabbitNamespace::Index::get() { return m_ns; } System::Collections::IEnumerator^ VowpalWabbitNamespace::EnumerableGetEnumerator::get() { return GetEnumerator(); } IEnumerator^ VowpalWabbitNamespace::GetEnumerator() { return gcnew FeatureEnumerator(m_example, m_features); } VowpalWabbitNamespace::FeatureEnumerator::FeatureEnumerator(VowpalWabbitExample^ example, features* features) : m_example(example), m_features(features), m_iterator(nullptr) { m_end = new Holder { features->end() }; } VowpalWabbitNamespace::FeatureEnumerator::~FeatureEnumerator() { delete m_end; delete m_iterator; } void VowpalWabbitNamespace::FeatureEnumerator::Reset() { delete m_iterator; m_iterator = nullptr; } bool VowpalWabbitNamespace::FeatureEnumerator::MoveNext() { if (m_iterator) ++m_iterator->value; else m_iterator = new Holder { m_features->begin() }; return m_iterator->value != m_end->value; } System::Object^ VowpalWabbitNamespace::FeatureEnumerator::IEnumeratorCurrent::get() { return Current; } VowpalWabbitFeature^ VowpalWabbitNamespace::FeatureEnumerator::Current::get() { if (!m_iterator || m_iterator->value == m_end->value) throw gcnew InvalidOperationException(); return gcnew VowpalWabbitFeature(m_example, m_iterator->value.value(), m_iterator->value.index()); } } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vw_example.h000066400000000000000000000147731323713302100210100ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #pragma once #include "vw_clr.h" #include "vw_interface.h" #include "vw_labelcomparator.h" #include "vw_label.h" namespace VW { using namespace System::Collections::Generic; using namespace VW::Labels; ref class VowpalWabbitExample; ref class VowpalWabbit; [System::Diagnostics::DebuggerDisplay("{m_weight_index}:{m_x}")] public ref struct VowpalWabbitFeature { private: feature_value m_x; uint64_t m_weight_index; VowpalWabbitExample^ m_example; VowpalWabbit^ m_vw; public: VowpalWabbitFeature(VowpalWabbitExample^ example, feature_value x, uint64_t weight_index); VowpalWabbitFeature(VowpalWabbit^ vw, feature_value x, uint64_t weight_index); property feature_value X { float get(); } property uint64_t FeatureIndex { uint64_t get(); } property uint64_t WeightIndex { uint64_t get(); } property float Weight { float get(); } property float AuditWeight { float get(); } virtual bool Equals(Object^ o) override; virtual int GetHashCode() override; }; template struct Holder { T value; }; [System::Diagnostics::DebuggerDisplay("{Index} = '{(char)Index}'")] public ref struct VowpalWabbitNamespace : public IEnumerable { private: ref class FeatureEnumerator : public IEnumerator { private: VowpalWabbitExample^ m_example; features* m_features; Holder* m_iterator; Holder* m_end; internal: FeatureEnumerator(VowpalWabbitExample^ example, features* features); ~FeatureEnumerator(); property System::Object^ IEnumeratorCurrent { virtual System::Object^ get() sealed = System::Collections::IEnumerator::Current::get; } public: virtual bool MoveNext(); virtual void Reset(); property VowpalWabbitFeature^ Current { virtual VowpalWabbitFeature^ get(); } }; namespace_index m_ns; features* m_features; VowpalWabbitExample^ m_example; property System::Collections::IEnumerator^ EnumerableGetEnumerator { virtual System::Collections::IEnumerator^ get() sealed = System::Collections::IEnumerable::GetEnumerator; } public: VowpalWabbitNamespace(VowpalWabbitExample^ m_example, namespace_index ns, features* features); ~VowpalWabbitNamespace(); property namespace_index Index { namespace_index get(); } virtual IEnumerator^ GetEnumerator(); }; /// /// A CLR representation of a vowpal wabbit example. /// /// /// Underlying memory is allocated by native code, but examples are not part of the ring. /// [System::Diagnostics::DebuggerDisplay("{m_example}: '{m_string}'")] public ref class VowpalWabbitExample : public IEnumerable { private: /// /// Reference to an optional underlying example. /// /// If this instance owns this is null. initonly VowpalWabbitExample^ m_innerExample; ref class NamespaceEnumerator : public IEnumerator { private: VowpalWabbitExample^ m_example; namespace_index* m_current; internal: NamespaceEnumerator(VowpalWabbitExample^ example); ~NamespaceEnumerator(); property System::Object^ IEnumeratorCurrent { virtual System::Object^ get() sealed = System::Collections::IEnumerator::Current::get; } public: virtual bool MoveNext(); virtual void Reset(); property VowpalWabbitNamespace^ Current { virtual VowpalWabbitNamespace^ get(); } }; protected: /// /// Returns native example data structure to owning instance. /// !VowpalWabbitExample(); internal: /// /// Initializes a new instance of . /// /// The parent instance. Examples cannot be shared between vw instances. /// The already allocated example structure VowpalWabbitExample(IVowpalWabbitExamplePool^ owner, example* example); /// /// The native example data structure. /// example* m_example; /// /// The owner of this example. /// IVowpalWabbitExamplePool^ m_owner; /// /// The optional string version of the example. /// String^ m_string; public: /// /// Initializes a new instance of . /// /// The parent instance. Examples cannot be shared between instances. /// The inner example this instance wraps. VowpalWabbitExample(IVowpalWabbitExamplePool^ owner, VowpalWabbitExample^ example); /// /// Returns native example data structure to owning pool. /// ~VowpalWabbitExample(); /// /// Extracts the prediction from this example using the given prediction factory. /// /// The prediction stored in this example. generic T GetPrediction(VowpalWabbit^ vw, IVowpalWabbitPredictionFactory^ factory); /// /// An optional inner example this example wraps. /// property VowpalWabbitExample^ InnerExample { VowpalWabbitExample^ get(); } /// /// The owner of this example. /// property IVowpalWabbitExamplePool^ Owner { IVowpalWabbitExamplePool^ get(); } /// /// The corresponding VowpalWabbitString for this example. /// property String^ VowpalWabbitString { String^ get(); void set(String^ value); } /// /// True if this is a new line example, otherwise false. /// /// A example without features is considered a new line example. property bool IsNewLine { bool get(); } String^ Diff(VowpalWabbit^ vw, VowpalWabbitExample^ other, IVowpalWabbitLabelComparator^ labelComparator); void MakeEmpty(VowpalWabbit^ vw); property System::Collections::IEnumerator^ EnumerableGetEnumerator { virtual System::Collections::IEnumerator^ get() sealed = System::Collections::IEnumerable::GetEnumerator; } virtual IEnumerator^ GetEnumerator(); property size_t NumberOfFeatures { size_t get(); } property ILabel^ Label { ILabel^ get(); void set(ILabel^ label); } }; } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vw_exception.cpp000066400000000000000000000013461323713302100216760ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #include "vw_clr.h" namespace VW { VowpalWabbitException::VowpalWabbitException(const vw_exception& ex) : Exception(gcnew System::String(ex.what())), m_filename(gcnew System::String(ex.Filename())), m_lineNumber(ex.LineNumber()) { } String^ VowpalWabbitException::Filename::get() { return m_filename; } Int32 VowpalWabbitException::LineNumber::get() { return m_lineNumber; } VowpalWabbitArgumentDisagreementException::VowpalWabbitArgumentDisagreementException(const vw_argument_disagreement_exception& ex) : VowpalWabbitException(ex) { } } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vw_interface.h000066400000000000000000000016351323713302100213060ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #pragma once #include "vw.h" #include "vw_clr.h" #include "vw_prediction.h" namespace VW { ref class VowpalWabbitExample; ref class VowpalWabbitBase; /// /// Owners of example must implement this interface. /// public interface class IVowpalWabbitExamplePool : public System::IDisposable { /// /// Gets or creates a new native examples. /// VowpalWabbitExample^ GetOrCreateNativeExample(); property VowpalWabbit^ Native { VowpalWabbit^ get(); } /// /// Puts a native example data structure back into the pool. /// /// The example to be returned. void ReturnExampleToPool(VowpalWabbitExample^ example); }; } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vw_label.h000066400000000000000000000170611323713302100204250ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #pragma once #include "vw.h" #include "vw_clr.h" #include "cb.h" #include "best_constant.h" #include "multiclass.h" namespace VW { namespace Labels { // The label classes are a replication of the parse_label function pointers provided by individual // modules. Main reason for creation is thread-saftey. The C++ version use a shared v_array in parser // and thus need to be synchronized. // These label classes are thread-safe and even more efficient as they avoid marshalling. using namespace System; using namespace System::Collections::Generic; using namespace System::Globalization; using namespace System::Text; using namespace CB; using namespace MULTICLASS; using namespace Newtonsoft::Json; public interface class ILabel { void UpdateExample(vw* vw, example* ex); void ReadFromExample(example* ex); }; public ref class ContextualBanditLabel sealed : ILabel { private: uint32_t m_action; float m_cost; float m_probability; public: ContextualBanditLabel() : m_action(0), m_cost(0), m_probability(0) { } ContextualBanditLabel(uint32_t action, float cost, float probability) : m_action(action), m_cost(cost), m_probability(0) { Probability = probability; } [JsonProperty] property uint32_t Action { uint32_t get() { return m_action; } void set(uint32_t value) { m_action = value; } } [JsonProperty] property float Probability { float get() { return m_probability; } void set(float value) { if (value < 0 || value >1) throw gcnew ArgumentOutOfRangeException("invalid probability: " + value); m_probability = value; } } [JsonProperty] property float Cost { float get() { return m_cost; } void set(float value) { m_cost = value; } } [JsonIgnore] property bool IsShared { bool get() { return m_cost == FLT_MAX && m_probability == -1.f; } } virtual void ReadFromExample(example* ex) { CB::label* ld = (CB::label*)&ex->l; if (ld->costs.size() > 0) { cb_class& f = ld->costs[0]; m_action = f.action; m_cost = f.cost; m_probability = f.probability; } } virtual void UpdateExample(vw* vw, example* ex) { CB::label* ld = (CB::label*)&ex->l; cb_class f; f.partial_prediction = 0.; f.action = m_action; f.cost = m_cost; f.probability = m_probability; ld->costs.push_back(f); } virtual String^ ToString() override { auto sb = gcnew StringBuilder; sb->Append(m_action.ToString(CultureInfo::InvariantCulture)); sb->Append(L':'); sb->Append(m_cost.ToString(CultureInfo::InvariantCulture)); sb->Append(L':'); sb->Append(m_probability.ToString(CultureInfo::InvariantCulture)); return sb->ToString(); } }; /// /// In multi-line scenarios the first example can contain a set of shared features. This first example must be /// marked using a 'shared' label. /// public ref class SharedLabel sealed : ILabel { private: uint32_t m_action; SharedLabel() : m_action((uint32_t)uniform_hash("shared", 6, 0)) { } public: static SharedLabel^ Instance = gcnew SharedLabel; virtual void UpdateExample(vw* vw, example* ex) { CB::label* ld = (CB::label*)&ex->l; cb_class f; f.partial_prediction = 0.; f.action = m_action; f.cost = FLT_MAX; f.probability = -1.f; ld->costs.push_back(f); } virtual String^ ToString() override { return "shared"; } virtual void ReadFromExample(example* ex) { } }; public ref class SimpleLabel sealed : ILabel { private: float m_label; Nullable m_weight; Nullable m_initial; public: SimpleLabel() : m_label(0) { } [JsonProperty] property float Label { float get() { return m_label; } void set(float value) { m_label = value; } } [JsonProperty(NullValueHandling = NullValueHandling::Ignore)] property Nullable Weight { Nullable get() { return m_weight; } void set(Nullable value) { m_weight = value; } } [JsonProperty(NullValueHandling = NullValueHandling::Ignore)] property Nullable Initial { Nullable get() { return m_initial; } void set(Nullable value) { m_initial = value; } } virtual void ReadFromExample(example* ex) { label_data* ld = (label_data*)&ex->l; m_label = ld->label; m_weight = ld->weight; m_initial = ld->initial; } virtual void UpdateExample(vw* vw, example* ex) { label_data* ld = (label_data*)&ex->l; ld->label = m_label; if (m_weight.HasValue) ld->weight = m_weight.Value; if (m_initial.HasValue) ld->initial = m_initial.Value; count_label(vw->sd, ld->label); } virtual String^ ToString() override { auto sb = gcnew StringBuilder; sb->Append(m_label.ToString(CultureInfo::InvariantCulture)); if (m_weight.HasValue) { sb->Append(L' '); sb->Append(m_weight.Value.ToString(CultureInfo::InvariantCulture)); if (m_initial.HasValue) { sb->Append(L' '); sb->Append(m_initial.Value.ToString(CultureInfo::InvariantCulture)); } } return sb->ToString(); } }; public ref class MulticlassLabel sealed : ILabel { public: ref class Label sealed { private: uint32_t m_class; Nullable m_weight; public: property uint32_t Class { uint32_t get() { return m_class; } void set(uint32_t value) { m_class = value; } } [JsonProperty(NullValueHandling = NullValueHandling::Ignore)] property Nullable Weight { Nullable get() { return m_weight; } void set(Nullable value) { m_weight = value; } } }; private: List^ m_classes; public: [JsonProperty] property List^ Classes { List^ get() { return m_classes; } void set(List^ value) { m_classes = value; } } virtual void ReadFromExample(example* ex) { throw gcnew NotImplementedException("to be done..."); } virtual void UpdateExample(vw* vw, example* ex) { throw gcnew NotImplementedException("to be done..."); } virtual String^ ToString() override { auto sb = gcnew StringBuilder; for each (Label^ label in m_classes) { sb->Append(L' '); sb->Append(label->Class.ToString(CultureInfo::InvariantCulture)); if (label->Weight.HasValue) { sb->Append(L' '); sb->Append(label->Weight.Value.ToString(CultureInfo::InvariantCulture)); } } // strip first space if (sb->Length > 0) sb->Remove(0, 1); return sb->ToString(); } }; public ref class StringLabel sealed : ILabel { private: String^ m_label; public: StringLabel() { } StringLabel(String^ label) : m_label(label) { } [JsonProperty] property String^ Label { String^ get() { return m_label; } void set(String^ value) { m_label = value; } } virtual void ReadFromExample(example* ex) { throw gcnew NotImplementedException("to be done..."); } virtual void UpdateExample(vw* vw, example* ex) { auto bytes = System::Text::Encoding::UTF8->GetBytes(m_label); auto valueHandle = GCHandle::Alloc(bytes, GCHandleType::Pinned); try { VW::parse_example_label(*vw, *ex, reinterpret_cast(valueHandle.AddrOfPinnedObject().ToPointer())); } CATCHRETHROW finally { valueHandle.Free(); } } virtual String^ ToString() override { return m_label; } }; } } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vw_labelcomparator.h000066400000000000000000000041571323713302100225170ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #pragma once #include "vw_example.h" namespace VW { ref class VowpalWabbitExample; /// /// Interface for label comparators. /// public interface class IVowpalWabbitLabelComparator { public: /// /// Compares labels of and . /// /// Returns null if labels are equivalent, otherwise returns the difference description. String^ Diff(VowpalWabbitExample^ ex1, VowpalWabbitExample^ ex2); }; /// /// A label comparer for simple labels. /// public ref class VowpalWabbitSimpleLabelComparator sealed : IVowpalWabbitLabelComparator { public: /// /// Compares labels of and . /// /// Returns null if labels are equivalent, otherwise returns the difference description. virtual String^ Diff(VowpalWabbitExample^ ex1, VowpalWabbitExample^ ex2) sealed; }; /// /// A label comparer for contextual bandit label. /// public ref class VowpalWabbitContextualBanditLabelComparator sealed : IVowpalWabbitLabelComparator { public: /// /// Compares labels of and . /// /// Returns null if labels are equivalent, otherwise returns the difference description. virtual String^ Diff(VowpalWabbitExample^ ex1, VowpalWabbitExample^ ex2) sealed; }; /// /// Label comparator factory. /// public ref class VowpalWabbitLabelComparator sealed abstract { public: /// /// Simple label comparator. /// static initonly IVowpalWabbitLabelComparator^ Simple = gcnew VowpalWabbitSimpleLabelComparator; /// /// Contextual bandit label comparator. /// static initonly IVowpalWabbitLabelComparator^ ContextualBandit = gcnew VowpalWabbitContextualBanditLabelComparator; }; } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vw_model.cpp000066400000000000000000000024131323713302100207740ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #include "vw_clr.h" #include "vw_model.h" #include "parse_regressor.h" #include "parse_args.h" #include "clr_io.h" namespace VW { VowpalWabbitSettings^ AddTestOnly(VowpalWabbitSettings^ settings) { // VowpalWabbitModel and VowpalWabbit instances seeded from VowpalWabbitModel // need to have the same "test" setting, otherwise the stride shift is different // and all hell breaks loose. if (!settings->Arguments->Contains("-t ") && !settings->Arguments->Contains("--testonly ") && !settings->Arguments->EndsWith("-t") && !settings->Arguments->EndsWith("--testonly")) { settings->Arguments += " -t"; } return settings; } VowpalWabbitModel::VowpalWabbitModel(VowpalWabbitSettings^ settings) : VowpalWabbitBase(AddTestOnly(settings)) { if (settings == nullptr) throw gcnew ArgumentNullException("settings"); if (settings->Model != nullptr) throw gcnew ArgumentNullException("VowpalWabbitModel cannot be initialized from another model"); } VowpalWabbitModel::VowpalWabbitModel(String^ args) : VowpalWabbitModel(gcnew VowpalWabbitSettings(args)) { } } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vw_model.h000066400000000000000000000014151323713302100204420ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #pragma once #include "vw_clr.h" #include "vw_base.h" namespace VW { /// /// VowpalWabbit model wrapper used in multi-threaded scenarios. /// public ref class VowpalWabbitModel : public VowpalWabbitBase { public: /// /// Initializes a new instance. /// /// Arguments passed to native instance. VowpalWabbitModel(VowpalWabbitSettings^ settings); /// Command line arguments passed to native instance. VowpalWabbitModel(String^ args); }; } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vw_prediction.cpp000066400000000000000000000121051323713302100220330ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #include "vw_prediction.h" #include "vw_example.h" #include "vw_base.h" #include "vowpalwabbit.h" namespace VW { void CheckExample(vw* vw, example* ex, prediction_type::prediction_type_t type) { if (vw == nullptr) throw gcnew ArgumentNullException("vw"); if (ex == nullptr) throw gcnew ArgumentNullException("ex"); auto ex_pred_type = vw->l->pred_type; if (ex_pred_type != type) { auto sb = gcnew StringBuilder(); sb->Append("Prediction type must be "); sb->Append(gcnew String(prediction_type::to_string(type))); sb->Append(" but is "); sb->Append(gcnew String(prediction_type::to_string(ex_pred_type))); throw gcnew ArgumentException(sb->ToString()); } } float VowpalWabbitScalarPredictionFactory::Create(vw* vw, example* ex) { CheckExample(vw, ex, PredictionType); try { return VW::get_prediction(ex); } CATCHRETHROW } VowpalWabbitScalar VowpalWabbitScalarConfidencePredictionFactory::Create(vw* vw, example* ex) { CheckExample(vw, ex, PredictionType); try { VowpalWabbitScalar ret; ret.Value = VW::get_prediction(ex); ret.Confidence = ex->confidence; return ret; } CATCHRETHROW } cli::array^ VowpalWabbitScalarsPredictionFactory::Create(vw* vw, example* ex) { CheckExample(vw, ex, PredictionType); try { auto& scalars = ex->pred.scalars; auto values = gcnew cli::array((int)scalars.size()); int index = 0; for (float s : scalars) values[index++] = s; return values; } CATCHRETHROW } float VowpalWabbitProbabilityPredictionFactory::Create(vw* vw, example* ex) { CheckExample(vw, ex, PredictionType); return ex->pred.prob; } float VowpalWabbitCostSensitivePredictionFactory::Create(vw* vw, example* ex) { CheckExample(vw, ex, PredictionType); try { return VW::get_cost_sensitive_prediction(ex); } CATCHRETHROW } Dictionary^ VowpalWabbitMulticlassProbabilitiesPredictionFactory::Create(vw* vw, example* ex) { #if _DEBUG if (ex == nullptr) throw gcnew ArgumentNullException("ex"); #endif v_array confidence_scores; try { confidence_scores = VW::get_cost_sensitive_prediction_confidence_scores(ex); } CATCHRETHROW auto values = gcnew Dictionary(); int i = 0; for (auto& val : confidence_scores) { values->Add(++i, val); } return values; } uint32_t VowpalWabbitMulticlassPredictionFactory::Create(vw* vw, example* ex) { CheckExample(vw, ex, PredictionType); return ex->pred.multiclass; } cli::array^ VowpalWabbitMultilabelPredictionFactory::Create(vw* vw, example* ex) { CheckExample(vw, ex, prediction_type::multilabels); size_t length; uint32_t* labels; try { labels = VW::get_multilabel_predictions(ex, length); } CATCHRETHROW if (length > Int32::MaxValue) throw gcnew ArgumentOutOfRangeException("Multi-label predictions too large"); auto values = gcnew cli::array((int)length); if (length > 0) Marshal::Copy(IntPtr(labels), values, 0, (int)length); return values; } cli::array^ VowpalWabbitActionScoreBasePredictionFactory::Create(vw* vw, example* ex) { CheckExample(vw, ex, PredictionType); auto& a_s = ex->pred.a_s; auto values = gcnew cli::array((int)a_s.size()); auto index = 0; for (auto& as : a_s) { values[index].Action = as.action; values[index].Score = as.score; index++; } return values; } cli::array^ VowpalWabbitTopicPredictionFactory::Create(vw* vw, example* ex) { if (ex == nullptr) throw gcnew ArgumentNullException("ex"); auto values = gcnew cli::array(vw->lda); Marshal::Copy(IntPtr(ex->pred.scalars.begin()), values, 0, vw->lda); return values; } System::Object^ VowpalWabbitDynamicPredictionFactory::Create(vw* vw, example* ex) { if (ex == nullptr) throw gcnew ArgumentNullException("ex"); switch (vw->l->pred_type) { case prediction_type::scalar: return VowpalWabbitPredictionType::Scalar->Create(vw, ex); case prediction_type::scalars: return VowpalWabbitPredictionType::Scalars->Create(vw, ex); case prediction_type::multiclass: return VowpalWabbitPredictionType::Multiclass->Create(vw, ex); case prediction_type::multilabels: return VowpalWabbitPredictionType::Multilabel->Create(vw, ex); case prediction_type::action_scores: return VowpalWabbitPredictionType::ActionScore->Create(vw, ex); case prediction_type::action_probs: return VowpalWabbitPredictionType::ActionProbabilities->Create(vw, ex); case prediction_type::prob: return VowpalWabbitPredictionType::Probability->Create(vw, ex); case prediction_type::multiclassprobs: return VowpalWabbitPredictionType::MultiClassProbabilities->Create(vw, ex); default: { auto sb = gcnew StringBuilder(); sb->Append("Unsupported prediction type: "); sb->Append(gcnew String(prediction_type::to_string(vw->l->pred_type))); throw gcnew ArgumentException(sb->ToString()); } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cli/vw_prediction.h000066400000000000000000000262331323713302100215070ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #pragma once #include "vw_clr.h" namespace VW { ref class VowpalWabbitExample; ref class VowpalWabbit; using namespace System::Collections::Generic; /// /// Interface for prediction factories enabling read-out of various prediction results in an extendable manner. /// generic public interface class IVowpalWabbitPredictionFactory { public: /// /// Creates a new prediction result from an example and the associated VW instance. /// /// A prediction result. /// Implementation must be thread-safe. T Create(vw* vw, example* ex); /// /// Returns the supported prediction type. /// property prediction_type::prediction_type_t PredictionType { prediction_type::prediction_type_t get(); } }; /// /// A scalar prediction result. /// public ref class VowpalWabbitDynamicPredictionFactory sealed : IVowpalWabbitPredictionFactory { public: /// /// Extracts prediction results from example. /// virtual System::Object^ Create(vw* vw, example* ex) sealed; /// /// Returns the supported prediction type. /// property prediction_type::prediction_type_t PredictionType { virtual prediction_type::prediction_type_t get() sealed { throw gcnew NotSupportedException("Prediction type is not available."); } } }; /// /// A scalar prediction result. /// public ref class VowpalWabbitScalarPredictionFactory sealed : IVowpalWabbitPredictionFactory { public: /// /// Extracts prediction results from example. /// virtual float Create(vw* vw, example* ex) sealed; /// /// Returns the supported prediction type. /// property prediction_type::prediction_type_t PredictionType { virtual prediction_type::prediction_type_t get() sealed { return prediction_type::scalar; } } }; public value struct VowpalWabbitScalar { public: float Value; float Confidence; }; /// /// A scalar prediction result. /// public ref class VowpalWabbitScalarConfidencePredictionFactory sealed : IVowpalWabbitPredictionFactory { public: /// /// Extracts prediction results from example. /// virtual VowpalWabbitScalar Create(vw* vw, example* ex) sealed; /// /// Returns the supported prediction type. /// property prediction_type::prediction_type_t PredictionType { virtual prediction_type::prediction_type_t get() sealed { return prediction_type::scalar; } } }; /// /// A scalar prediction result. /// public ref class VowpalWabbitScalarsPredictionFactory sealed : IVowpalWabbitPredictionFactory^> { public: /// /// Extracts prediction results from example. /// virtual cli::array^ Create(vw* vw, example* ex) sealed; /// /// Returns the supported prediction type. /// property prediction_type::prediction_type_t PredictionType { virtual prediction_type::prediction_type_t get() sealed { return prediction_type::scalars; } } }; /// /// A scalar prediction result. /// public ref class VowpalWabbitProbabilityPredictionFactory sealed : IVowpalWabbitPredictionFactory { public: /// /// Extracts prediction results from example. /// virtual float Create(vw* vw, example* ex) sealed; /// /// Returns the supported prediction type. /// property prediction_type::prediction_type_t PredictionType { virtual prediction_type::prediction_type_t get() sealed { return prediction_type::prob; } } }; /// /// A cost sensitive prediction result. /// public ref class VowpalWabbitCostSensitivePredictionFactory sealed : IVowpalWabbitPredictionFactory { public: /// /// Extracts cost sensitive prediction results from example. /// virtual float Create(vw* vw, example* ex) sealed; /// /// Returns the supported prediction type. /// property prediction_type::prediction_type_t PredictionType { virtual prediction_type::prediction_type_t get() sealed { return prediction_type::multiclass; } } }; /// /// A cost sensitive prediction result. /// public ref class VowpalWabbitMulticlassPredictionFactory sealed : IVowpalWabbitPredictionFactory { public: /// /// Extracts cost sensitive prediction results from example. /// virtual uint32_t Create(vw* vw, example* ex) sealed; /// /// Returns the supported prediction type. /// property prediction_type::prediction_type_t PredictionType { virtual prediction_type::prediction_type_t get() sealed { return prediction_type::multiclass; } } }; /// /// A cost sensitive prediction result with associated confidence score /// For -oaa --probabilities /// public ref class VowpalWabbitMulticlassProbabilitiesPredictionFactory sealed : IVowpalWabbitPredictionFactory^> { public: /// /// Extracts cost sensitive prediction results from example, including confidence score. /// virtual Dictionary^ Create(vw* vw, example* ex) sealed; /// /// Returns the supported prediction type. /// property prediction_type::prediction_type_t PredictionType { virtual prediction_type::prediction_type_t get() sealed { return prediction_type::multiclassprobs; } } }; /// /// A multi label prediction result. /// public ref class VowpalWabbitMultilabelPredictionFactory sealed : IVowpalWabbitPredictionFactory^> { public: /// /// Extracts multilabel prediction results from example. /// virtual cli::array^ Create(vw* vw, example* ex) sealed; /// /// Returns the supported prediction type. /// property prediction_type::prediction_type_t PredictionType { virtual prediction_type::prediction_type_t get() sealed { return prediction_type::multilabels; } } }; [System::Diagnostics::DebuggerDisplay("{Action}:{Score}")] public value struct ActionScore sealed { public: property uint32_t Action; property float Score; }; /// /// A action score/probability result. /// public ref class VowpalWabbitActionScoreBasePredictionFactory abstract : IVowpalWabbitPredictionFactory^> { public: /// /// Extracts multilabel prediction results from example. /// virtual cli::array^ Create(vw* vw, example* ex) sealed; /// /// Returns the supported prediction type. /// property prediction_type::prediction_type_t PredictionType { virtual prediction_type::prediction_type_t get() abstract; } }; /// /// A action score prediction result. /// public ref class VowpalWabbitActionScorePredictionFactory sealed : public VowpalWabbitActionScoreBasePredictionFactory { public: /// /// Returns the supported prediction type. /// property prediction_type::prediction_type_t PredictionType { virtual prediction_type::prediction_type_t get() override sealed { return prediction_type::action_scores; } } }; /// /// A multi label prediction result. /// public ref class VowpalWabbitActionProbabilitiesPredictionFactory sealed : public VowpalWabbitActionScoreBasePredictionFactory { public: /// /// Returns the supported prediction type. /// property prediction_type::prediction_type_t PredictionType { virtual prediction_type::prediction_type_t get() override sealed { return prediction_type::action_probs; } } }; /// /// A topic prediction result. /// public ref class VowpalWabbitTopicPredictionFactory sealed : IVowpalWabbitPredictionFactory^> { public: /// /// Extracts prediction results from example. The predicted topics. /// virtual cli::array^ Create(vw* vw, example* ex) sealed; /// /// Returns the supported prediction type. /// property prediction_type::prediction_type_t PredictionType { virtual prediction_type::prediction_type_t get() sealed { throw gcnew NotSupportedException("Prediction type is not available."); } } }; /// /// Provides convenient collection of all prediction types. /// public ref class VowpalWabbitPredictionType sealed abstract { public: /// /// Use for scalar predictions. /// static initonly VowpalWabbitScalarPredictionFactory^ Scalar = gcnew VowpalWabbitScalarPredictionFactory; /// /// Use for scalar predictions. /// static initonly VowpalWabbitScalarConfidencePredictionFactory^ ScalarConfidence = gcnew VowpalWabbitScalarConfidencePredictionFactory; /// /// Use for scalar predictions. /// static initonly VowpalWabbitScalarsPredictionFactory^ Scalars = gcnew VowpalWabbitScalarsPredictionFactory; /// /// Use for cost sensitive predictions. /// static initonly VowpalWabbitCostSensitivePredictionFactory^ CostSensitive = gcnew VowpalWabbitCostSensitivePredictionFactory; /// /// Use for multi label predictions. /// static initonly VowpalWabbitMultilabelPredictionFactory^ Multilabel = gcnew VowpalWabbitMultilabelPredictionFactory; /// /// Use for multi class predictions. /// static initonly VowpalWabbitMulticlassPredictionFactory^ Multiclass = gcnew VowpalWabbitMulticlassPredictionFactory; /// /// Use for action score predictions. /// static initonly VowpalWabbitActionScorePredictionFactory^ ActionScore = gcnew VowpalWabbitActionScorePredictionFactory; /// /// Use for action score predictions. /// static initonly VowpalWabbitActionProbabilitiesPredictionFactory^ ActionProbabilities = gcnew VowpalWabbitActionProbabilitiesPredictionFactory; /// /// Use for LDA topic predictions. /// static initonly VowpalWabbitTopicPredictionFactory^ Topic = gcnew VowpalWabbitTopicPredictionFactory; /// /// Use for dynamicially determined predictions. /// static initonly VowpalWabbitDynamicPredictionFactory^ Dynamic = gcnew VowpalWabbitDynamicPredictionFactory; /// /// Use for dynamicially determined predictions. /// static initonly VowpalWabbitProbabilityPredictionFactory^ Probability = gcnew VowpalWabbitProbabilityPredictionFactory; /// /// Use for multiclass predictions with probabilities /// static initonly VowpalWabbitMulticlassProbabilitiesPredictionFactory^ MultiClassProbabilities = gcnew VowpalWabbitMulticlassProbabilitiesPredictionFactory; }; }vowpal-wabbit-8.5.0.dfsg1/cs/cli/vw_settings.h000066400000000000000000000077311323713302100212110ustar00rootroot00000000000000/* Copyright (c) by respective owners including Yahoo!, Microsoft, and individual contributors. All rights reserved. Released under a BSD (revised) license as described in the file LICENSE. */ #pragma once #include using namespace System; using namespace System::Collections::Generic; using namespace System::IO; using namespace System::Threading::Tasks; using namespace VW::Serializer; namespace VW { ref class VowpalWabbit; ref class VowpalWabbitModel; ref class VowpalWabbitSettings; public enum class VowpalWabbitExampleDistribution { /// /// Statistically safer option. /// UniformRandom = 0, /// /// Better runtime performance. /// RoundRobin = 1 }; public interface class ITypeInspector { public: Schema^ CreateSchema(VowpalWabbitSettings^ settings, Type^ type); }; /// /// Settings for wrapper. /// /// Constructor with optional arguments was dropped as it broke version remapping (signature changed with the introduction of new options). public ref class VowpalWabbitSettings : public ICloneable { public: VowpalWabbitSettings() { Arguments = String::Empty; ExampleCountPerRun = 1000; MaxExampleCacheSize = UINT32_MAX; MaxExampleQueueLengthPerInstance = UINT32_MAX; EnableExampleCaching = false; // default to the statistically more safe option ExampleDistribution = VowpalWabbitExampleDistribution::UniformRandom; EnableStringExampleGeneration = false; EnableStringFloatCompact = false; PropertyConfiguration = ::PropertyConfiguration::Default; EnableThreadSafeExamplePooling = false; MaxExamples = INT32_MAX; Verbose = false; } VowpalWabbitSettings(String^ arguments) : VowpalWabbitSettings() { if (arguments != nullptr) Arguments = arguments; } /// /// Command line arguments. /// property String^ Arguments; /// /// Model used for initialization. /// property Stream^ ModelStream; /// /// Shared native vowpwal wabbit data structure. /// property VowpalWabbitModel^ Model; property ParallelOptions^ ParallelOptions; /// /// Set to true to disable example caching when used with a serializer. Defaults to true. /// property bool EnableExampleCaching; /// /// Maximum number of serialized examples cached. Defaults to UINT32_MAX. /// property uint32_t MaxExampleCacheSize; /// /// Maximum number of examples accepted by VowpalWabbitManager until Learn/Predict/... start to block. Defaults to UINT32_MAX. /// property uint32_t MaxExampleQueueLengthPerInstance; property uint32_t Node; property VowpalWabbit^ Root; property VowpalWabbitExampleDistribution ExampleDistribution; /// /// In multi-threaded mode, this is the number of examples processed per run. /// After ecah run the models are synchronized. /// Defaults to 1000. /// property uint32_t ExampleCountPerRun; /// /// Enable Vowpal Wabbit native string generation. /// property bool EnableStringExampleGeneration; /// /// Enable compact float serialization for Vowpal Wabbit native string generation. /// property bool EnableStringFloatCompact; property VW::Serializer::Schema^ Schema; property VW::Serializer::Schema^ ActionDependentSchema; property List^ CustomFeaturizer; property ITypeInspector^ TypeInspector; property PropertyConfiguration^ PropertyConfiguration; property bool EnableThreadSafeExamplePooling; property int MaxExamples; property bool Verbose; /// /// Action invoked for each trace message. /// /// /// The trace listener obeys the Verbose property, which defaults to false. /// property Action^ TraceListener; virtual Object^ Clone() { return MemberwiseClone(); } }; } vowpal-wabbit-8.5.0.dfsg1/cs/common/000077500000000000000000000000001323713302100171755ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/common/Bag.cs000066400000000000000000000120011323713302100202070ustar00rootroot00000000000000using System; using System.Collections.Concurrent; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading; using System.Threading.Tasks; namespace VW { /// /// Interface to model simple bag. /// /// public interface IBag { /// /// Try add to this bag. /// /// The item to add. /// True if succesful, false otherwise. bool TryAdd(T item); /// /// Remove and return one item from this bag. /// /// The item removed from the bag or default(T) if there is no item available. T Remove(); /// /// Remove and return all items from this bag. /// /// The items removed from the gag. IEnumerable RemoveAll(); /// /// The number of items this bag contains. /// int Count { get; } } /// /// Factory class for various bag implementations. /// public static class Bag { /// /// Creates a simple bound or unbound, not thread-safe bag object. /// /// The type of the items. /// The maximum number of items this bag should hold. /// A new bag instance. public static IBag Create(int max = int.MaxValue) { return max == int.MaxValue ? (IBag)new BagImpl() : new BoundedBagImpl(max); } /// /// Creates an unbound thread-safe, lock free bag. /// /// The type of the items. /// A new bag instance. public static IBag CreateLockFree(int max = int.MaxValue) { return new LockFreeBagImpl(max); } private abstract class BaseBagImpl { protected readonly Stack stack; internal BaseBagImpl() { this.stack = new Stack(); } public T Remove() { return this.stack.Count == 0 ? default(T) : this.stack.Pop(); } public IEnumerable RemoveAll() { var ret = this.stack.ToArray(); this.stack.Clear(); return ret; } public int Count { get { return this.stack.Count; } } } private sealed class BagImpl : BaseBagImpl, IBag { public bool TryAdd(T item) { this.stack.Push(item); return true; } } private sealed class BoundedBagImpl : BaseBagImpl, IBag { private readonly int max; internal BoundedBagImpl(int max) { this.max = max; } public bool TryAdd(T item) { if (this.stack.Count >= this.max) return false; this.stack.Push(item); return true; } } /// /// This is a good read on performance: http://msdn.microsoft.com/en-us/concurrency/ee851578.aspx /// For streaming training we are seeking good performance for a single producer and multiple consumers. /// /// private sealed class LockFreeBagImpl : IBag { private readonly int max; private readonly ConcurrentQueue queue; private int count; internal LockFreeBagImpl(int max) { this.queue = new ConcurrentQueue(); this.max = max; } public bool TryAdd(T item) { if (this.count < this.max) { this.queue.Enqueue(item); Interlocked.Increment(ref this.count); return true; } return false; } public T Remove() { T result; if (this.queue.TryDequeue(out result)) { Interlocked.Decrement(ref this.count); return result; } return default(T); } public IEnumerable RemoveAll() { // TODO: violates the lock constraint. though this is just used at disposable time return this.queue; } public int Count { get { return this.count; } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/common/Properties/000077500000000000000000000000001323713302100213315ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/common/Properties/AssemblyInfo.cs000066400000000000000000000022311323713302100242510ustar00rootroot00000000000000//------------------------------------------------------------------------------ // // This code was generated by a tool. // Runtime Version:4.0.30319.42000 // // Changes to this file may cause incorrect behavior and will be lost if // the code is regenerated. // //------------------------------------------------------------------------------ [assembly: System.Reflection.AssemblyTitle("Vowpal Wabbit Common")] [assembly: System.Reflection.AssemblyDescription("Vowpal Wabbit Common")] [assembly: System.Reflection.AssemblyCompany("Microsoft Corp")] [assembly: System.Reflection.AssemblyProduct("Vowpal Wabbit")] [assembly: System.Reflection.AssemblyCopyright("Copyright (C) Microsoft Corp 2012-2016, Yahoo! Inc. 2007-2012, and many individua" + "l contributors. All rights reserved")] [assembly: System.Runtime.InteropServices.ComVisible(false)] [assembly: System.CLSCompliant(false)] [assembly: System.Runtime.InteropServices.Guid("091c7906-1f69-44d5-a15f-fb29847a68ef")] [assembly: System.Reflection.AssemblyVersion("8.4.0.1")] [assembly: System.Reflection.AssemblyFileVersion("8.4.0.1")] vowpal-wabbit-8.5.0.dfsg1/cs/common/Reflection/000077500000000000000000000000001323713302100212675ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/common/Reflection/InspectionHelper.cs000066400000000000000000000060011323713302100250660ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Diagnostics.Contracts; using System.Linq; namespace VW.Reflection { /// /// Utilitiy class supporting feature type inspection. /// public static class InspectionHelper { /// /// Determines if the is a supported numeric type. /// /// The type to be inspected. /// True if numeric, false otherwise. public static bool IsNumericType(Type elemType) { return IsNumericTypeInternal(elemType) || (elemType != null && elemType.IsGenericType && elemType.GetGenericTypeDefinition() == typeof(Nullable<>) && IsNumericTypeInternal(elemType.GetGenericArguments()[0])); } private static bool IsNumericTypeInternal(Type elemType) { return elemType == typeof(double) || elemType == typeof(float) || elemType == typeof(byte) || elemType == typeof(sbyte) || elemType == typeof(char) || elemType == typeof(decimal) || elemType == typeof(UInt16) || elemType == typeof(UInt32) || elemType == typeof(UInt64) || elemType == typeof(Int16) || elemType == typeof(Int32) || elemType == typeof(Int64); } /// /// If is an enumerable type (such as array or ), this method will /// return the element type. /// /// The type to be inspected. /// If is an enumerable type the element type is returned, otherwise null. public static Type GetEnumerableElementType(Type type) { Contract.Requires(type != null); if (type.IsArray) return type.GetElementType(); var enumerableType = type.GetInterfaces().Union(new[] { type }) .FirstOrDefault(it => it.IsGenericType && it.GetGenericTypeDefinition() == typeof(IEnumerable<>)); // let's get T of IEnumerable if (enumerableType != null) return enumerableType.GetGenericArguments()[0]; return null; } } } vowpal-wabbit-8.5.0.dfsg1/cs/common/Reflection/ReflectionHelper.cs000066400000000000000000000307731323713302100250620ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Diagnostics; using System.Diagnostics.Contracts; using System.IO; using System.Linq; using System.Linq.Expressions; using System.Reflection; using System.Reflection.Emit; namespace VW.Reflection { /// /// Reflection helper to find methods on visitors. /// public static class ReflectionHelper { /// /// Compiles the supplied to a callable function. /// /// The source expression to be compiled. /// A callable function. /// Can't constraint on Func (or would have to have 11 overloads) nor is it possible to constaint on delegate. public static System.Delegate CompileToFunc(this Expression sourceExpression) { // inspect T to be Func<...> var funcType = typeof(T); if (!funcType.Name.StartsWith("Func`")) throw new ArgumentException("T must be one of the System.Func<...> type."); var genericArguments = funcType.GetGenericArguments(); var returnType = genericArguments.Last(); var paramTypes = genericArguments.Take(genericArguments.Length - 1); // sign serializer so we can get access to internal members. var asmName = new AssemblyName("VowpalWabbitSerializer"); StrongNameKeyPair kp; using (var stream = typeof(ReflectionHelper).Assembly.GetManifestResourceStream("VW.vw_key.snk")) using (var memStream = new MemoryStream()) { stream.CopyTo(memStream, 1024); kp = new StrongNameKeyPair(memStream.ToArray()); } asmName.KeyPair = kp; var dynAsm = AppDomain.CurrentDomain.DefineDynamicAssembly(asmName, AssemblyBuilderAccess.RunAndSave); // Create a dynamic module and type //#if !DEBUG //var moduleBuilder = dynAsm.DefineDynamicModule("VowpalWabbitSerializerModule", asmName.Name + ".dll", true); //#else var moduleBuilder = dynAsm.DefineDynamicModule("VowpalWabbitSerializerModule"); var typeBuilder = moduleBuilder.DefineType("VowpalWabbitSerializer" + Guid.NewGuid().ToString().Replace('-', '_')); // Create our method builder for this type builder const string methodName = "Method"; var methodBuilder = typeBuilder.DefineMethod( methodName, MethodAttributes.Public | MethodAttributes.Static, returnType, paramTypes.ToArray()); // compared to Compile this looks rather ugly, but there is a feature-bug // that adds a security check to every call of the Serialize method //#if !DEBUG //var debugInfoGenerator = DebugInfoGenerator.CreatePdbGenerator(); //visit.CompileToMethod(methodBuilder, debugInfoGenerator); //#else sourceExpression.CompileToMethod(methodBuilder); //#endif var dynType = typeBuilder.CreateType(); // for debugging only // dynAsm.Save(@"my.dll"); return Delegate.CreateDelegate(typeof(T), dynType.GetMethod(methodName)); } /// /// TODO: replace me with Roslyn once it's released and just generate string code. This way the overload resolution is properly done. /// /// This is a simple heuristic for overload resolution, not the full thing. public static MethodInfo FindMethod(Type objectType, string name, params Type[] parameterTypes) { Contract.Requires(objectType != null); Contract.Requires(name != null); Contract.Requires(parameterTypes != null); // let's find the "best" match: // order by // 1. distance (0 = assignable, 1 = using generic) --> ascending // 2. # of interfaces implemented. the more the better (the more specific we are) --> descending // 3. # of open generics. the less the better (the more specific we are) --> ascending var methods = from m in objectType.GetMethods(BindingFlags.Instance | BindingFlags.Public) where m.Name == name let parameters = m.GetParameters() where parameters.Length == parameterTypes.Length let output = parameterTypes.Zip(parameters, (valueType, methodParameter) => Distance(methodParameter.ParameterType, valueType)).ToArray() where output.All(o => o != null) let distance = output.Sum(o => o.Distance) let interfacesImplemented = output.Sum(o => o.InterfacesImplemented) let generics = output.Sum(o => o.GenericTypes.Count) orderby distance, generics, interfacesImplemented descending select new { Method = m, Distance = distance, InterfacesImplemented = interfacesImplemented, GenericTypes = output.Select(o => o.GenericTypes) }; var bestCandidate = methods.FirstOrDefault(); if (bestCandidate == null) { return null; } MethodInfo method = bestCandidate.Method; //Debug.WriteLine("Method Search"); //foreach (var item in methods) //{ // Debug.WriteLine(string.Format("Distance={0} Interfaces={1} OpenGenerics={2} Method={3}", // item.Distance, // item.InterfacesImplemented, // item.GenericTypes.Count(gt => gt.Count > 0), // item.Method)); //} if (method.IsGenericMethod) { var mergedGenericTypes = bestCandidate.GenericTypes.SelectMany(d => d).ToLookup(kvp => kvp.Key, kvp => kvp.Value); // consistency check foreach (var gt in mergedGenericTypes) { var refElem = gt.First(); if (gt.Any(t => t != refElem)) { throw new NotSupportedException("Inconsistent generic argument mapping: " + string.Join(",", gt)); } } // map generic arguments to actual argument var actualTypes = method.GetGenericArguments().Select(t => mergedGenericTypes[t].First()).ToArray(); method = method.MakeGenericMethod(actualTypes); //Debug.WriteLine("\t specializing: " + method); } // Debug.WriteLine("Method: {0} for {1} {2}", method, name, string.Join(",", parameterTypes.Select(t => t.ToString()))); return method; } internal static TypeMatch Distance(Type candidate, Type valueType) { if (candidate == valueType) { return new TypeMatch(0) { InterfacesImplemented = candidate.GetInterfaces().Count() }; } if (candidate.IsAssignableFrom(valueType)) { return new TypeMatch(1) { InterfacesImplemented = candidate.GetInterfaces().Count() }; } if (candidate.IsGenericParameter && candidate.GetGenericParameterConstraints().All(c => c.IsAssignableFrom(valueType))) { return new TypeMatch(2, candidate, valueType) { InterfacesImplemented = candidate.GetInterfaces().Count() }; } if (candidate.IsGenericType) { // try to find a match that is assignable... // var genericCandidate = candidate.GetGenericTypeDefinition(); var bestMatches = from typeDistance in valueType.GetInterfaces().Select(it => new TypeDistance { Distance = 1, Type = it }) .Union(GetBaseTypes(valueType)) let type = typeDistance.Type where type.IsGenericType && type.GetGenericTypeDefinition() == genericCandidate let distances = candidate.GetGenericArguments().Zip(type.GetGenericArguments(), (a, b) => Distance(a, b)).ToList() where distances.All(d => d != null) let output = new TypeMatch(typeDistance.Distance, distances) { InterfacesImplemented = distances.Sum(d => d.InterfacesImplemented) + (candidate.IsInterface ? candidate.GetInterfaces().Count() : 0) } orderby output.Distance, output.InterfacesImplemented descending, output.GenericTypes.Count select output; return bestMatches.FirstOrDefault(); } return null; } internal static IEnumerable GetBaseTypes(Type type, int depth = 0) { if (type == typeof(object) || type == null) { yield break; } yield return new TypeDistance { Type = type, Distance = depth }; foreach (var item in GetBaseTypes(type.BaseType, depth + 1)) { yield return item; } } /// /// Gets the member info in a sort of type safe manner - it's better than using strings, but some runtime errors are still possbile. /// public static MemberInfo GetInfo(Expression> expression) { Contract.Requires(expression != null); return GetInfo(expression.Body); } /// /// Gets the member info in a sort of type safe manner - it's better than using strings, but some runtime errors are still possbile. /// public static MemberInfo GetInfo(Expression> expression) { Contract.Requires(expression != null); return GetInfo(expression.Body); } /// /// Gets the member info in a sort of type safe manner - it's better than using strings, but some runtime errors are still possbile. /// public static MemberInfo GetInfo(Expression expression) { Contract.Requires(expression != null); var binaryExpression = expression as BinaryExpression; if (binaryExpression != null) { if (binaryExpression.Method != null) { return binaryExpression.Method; } throw new NotSupportedException(); } var methodExpression = expression as MemberExpression; if (methodExpression != null) { return methodExpression.Member; } var methodCallExpression = expression as MethodCallExpression; if (methodCallExpression != null) { return methodCallExpression.Method; } var newExpression = expression as NewExpression; if (newExpression != null) { return newExpression.Constructor; } var unaryExpression = expression as UnaryExpression; if (unaryExpression != null) { if (unaryExpression.Method != null) { return unaryExpression.Method; } } throw new NotSupportedException(); } } } vowpal-wabbit-8.5.0.dfsg1/cs/common/Reflection/TypeDistance.cs000066400000000000000000000013571323713302100242200ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; namespace VW.Reflection { /// /// Models a distance to a given type. /// internal sealed class TypeDistance { internal int Distance { get; set; } internal Type Type { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/common/Reflection/TypeMatch.cs000066400000000000000000000035461323713302100235240ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Diagnostics.Contracts; using System.Linq; namespace VW.Reflection { /// /// Models a distance and additional matching constraints for a type. /// internal sealed class TypeMatch { internal TypeMatch(int distance) { this.Distance = distance; this.GenericTypes = new Dictionary { }; } internal TypeMatch(int distance, Type genericType, Type actualType) : this(distance) { Contract.Requires(genericType != null); Contract.Requires(actualType != null); this.GenericTypes = new Dictionary { { genericType, actualType } }; } internal TypeMatch(int distance, IEnumerable typeMatches) : this(distance) { Contract.Requires(typeMatches != null); this.GenericTypes = typeMatches .Where(tm => tm.GenericTypes != null) .SelectMany(tm => tm.GenericTypes) .ToDictionary(tm => tm.Key, tm => tm.Value); } internal int Distance { get; private set; } internal int InterfacesImplemented { get; set; } internal IDictionary GenericTypes { get; private set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/common/Serializer/000077500000000000000000000000001323713302100213065ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/common/Serializer/FeatureExpression.cs000066400000000000000000000202631323713302100253130ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Diagnostics; using System.Diagnostics.Contracts; using System.Linq.Expressions; using System.Reflection; using VW.Reflection; namespace VW.Serializer { /// /// Delegate defintion for feature object creation expressions. /// /// An expression resolving to a VowpalWabbit instance. /// An expression resolving to a Namespace instance. /// An expression constructing a new Feature object. public delegate Expression NewFeatureExpressionDelegate(Expression vw, Expression @namespace); /// /// Feature data composed during compilation step. /// [DebuggerDisplay("FeatureExpression({Name})")] public sealed class FeatureExpression { /// /// Initializes a new instance of the class. /// /// The type of the feature. /// The name of the feature. /// Factory to extract the value for a given feature from the example object (input argument). /// Factories to provide validation before invoking the expression created through . /// The expression must create new Feature instances. /// The namespace this feature belongs to. /// The feature group this feature belongs to. /// If true the marshaller enumerates the feature (as in creates a 1-hot encoding). /// The variable name to be used in the generated code. /// Used to order feature serialization. /// True if an anchor element should be added at the beginning of a dense feature array. /// Configures string pre-processing for this feature. /// An optional method overriding the otherwise auto-resolved serialization method. /// True if a dictionary should be build for this feature. /// The parent feature expression. public FeatureExpression(Type featureType, string name, Func valueExpressionFactory, List> valueValidExpressionFactories = null, NewFeatureExpressionDelegate featureExpressionFactory = null, string @namespace = null, char? featureGroup = null, bool enumerize = false, string variableName = null, int? order = null, bool addAnchor = false, StringProcessing stringProcessing = StringProcessing.Split, MethodInfo overrideSerializeMethod = null, bool? dictify = null, FeatureExpression parent = null) { if (featureType == null) throw new ArgumentNullException("featureType"); if (valueExpressionFactory == null) throw new ArgumentNullException("valueExpressionFactory"); Contract.EndContractBlock(); if(featureType.IsGenericType && featureType.GetGenericTypeDefinition() == typeof(Nullable<>)) { this.IsNullable = true; this.FeatureType = featureType.GetGenericArguments()[0]; } else { this.IsNullable = false; this.FeatureType = featureType; } this.Name = name; this.ValueExpressionFactory = valueExpressionFactory; this.ValueValidExpressionFactories = valueValidExpressionFactories; this.FeatureExpressionFactory = featureExpressionFactory; this.Namespace = @namespace; this.FeatureGroup = featureGroup; this.Enumerize = enumerize; this.VariableName = variableName ?? name; this.Order = order ?? 1; this.AddAnchor = addAnchor; this.Dictify = dictify ?? false; this.StringProcessing = stringProcessing; this.OverrideSerializeMethod = overrideSerializeMethod; this.Dictify = dictify ?? false; this.Parent = parent; this.DenseFeatureValueElementType = InspectionHelper.GetEnumerableElementType(featureType); if (!InspectionHelper.IsNumericType(this.DenseFeatureValueElementType)) this.DenseFeatureValueElementType = null; } /// /// The parent feature expression. /// public FeatureExpression Parent { get; private set; } /// /// True if the type is nullable. /// public bool IsNullable { get; set; } /// /// Serializer variable name. /// /// Useful to debug public string VariableName { get; set; } /// /// The type of the feature. /// public Type FeatureType { get; private set; } internal Type IntermediateFeatureType { get; set; } /// /// The name of the feature. /// public string Name { get; set; } /// /// The namespace. /// public string Namespace { get; set; } /// /// The feature group. /// public char? FeatureGroup { get; set; } /// /// An optional method overriding the otherwise auto-resolved serialization method. /// public MethodInfo OverrideSerializeMethod { get; set; } /// /// True if this feature should be enumerized. /// public bool Enumerize { get; set; } /// /// True if an anchor element should be added at the beginning of a dense feature array. /// public bool AddAnchor { get; set; } /// /// True if a dictionary should be build for this feature. /// public bool Dictify { get; set; } /// /// Factory to extract the value for a given feature from the example object (input argument). /// public Func ValueExpressionFactory { get; set; } /// /// Factories to provide validation before invoking the expression created through . /// public List> ValueValidExpressionFactories { get; set; } /// /// The expression must create new Feature instances. /// public NewFeatureExpressionDelegate FeatureExpressionFactory { get; set; } /// /// The element type of an enumerable feature type. /// public Type DenseFeatureValueElementType { get; set; } /// /// Used to order feature serialization. /// public int Order { get; set; } /// /// Configures string pre-processing for this feature. /// public StringProcessing StringProcessing { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/common/Serializer/LabelExpression.cs000066400000000000000000000027471323713302100247460ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Linq.Expressions; namespace VW.Serializer { /// /// Defines access to the label for an user-specified example type. /// public sealed class LabelExpression { /// /// The name of the label. /// public string Name { get; set; } /// /// The type of the feature. /// public Type LabelType { get; set; } /// /// Factory to extract the value for a given feature from the example object (input argument). /// public Func ValueExpressionFactory { get; set; } /// /// Factories to provide validation before invoking the expression created through . /// public List> ValueValidExpressionFactories { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/common/Serializer/PropertyConfiguration.cs000066400000000000000000000105351323713302100262150ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; namespace VW.Serializer { /// /// Constants used throughout C# wrapper. /// public sealed class PropertyConfiguration { /// /// Default value for feature ignore prefix: '_'. /// public const string FeatureIgnorePrefixDefault = "_"; /// /// Default value for text property: '_text'. /// public const string TextPropertyDefault = "_text"; /// /// Default value for label property: '_label'. /// public const string LabelPropertyDefault = "_label"; /// /// Default value for label index property: '_labelindex'. /// public const string LabelIndexPropertyDefault = "_labelindex"; /// /// Default value for label property prefix: '_label_'; /// public const string LabelPropertyPrefixDefault = "_label_"; /// /// Default value for multi property: '_multi'. /// public const string MultiPropertyDefault = "_multi"; /// /// Default singleton holding the default configuration. /// public static readonly PropertyConfiguration Default = new PropertyConfiguration(); /// /// Initializes a new instance. /// public PropertyConfiguration() { this.FeatureIgnorePrefix = FeatureIgnorePrefixDefault; this.TextProperty = TextPropertyDefault; this.LabelProperty = LabelPropertyDefault; this.MultiProperty = MultiPropertyDefault; this.LabelIndexProperty = LabelIndexPropertyDefault; this.LabelPropertyPrefix = LabelPropertyPrefixDefault; } /// /// JSON properties starting with underscore are ignored. /// public string FeatureIgnorePrefix { get; set; } /// /// JSON property "_text" is marshalled using . /// public string TextProperty { get; set; } /// /// JSON property "_label" is used as label. /// public string LabelProperty { get; set; } /// /// JSON property "_labelIndex" determines the index this label is applied for multi-line examples. /// public string LabelIndexProperty { get; set; } /// /// JSON properties starting with "_label_$name" are used to specify nested properties. Has the same effect as _label: { "$name": ... }. /// public string LabelPropertyPrefix { get; set; } /// /// JSON property "_multi" is used to signal multi-line examples. /// public string MultiProperty { get; set; } /// /// True if is considered a special property and thus should not be skipped. /// /// The JSON property name. /// True if is a special property, false otherwise. public bool IsSpecialProperty(string property) { return property.Equals(TextProperty, StringComparison.OrdinalIgnoreCase) || property.Equals(LabelProperty, StringComparison.OrdinalIgnoreCase) || property.Equals(MultiProperty, StringComparison.OrdinalIgnoreCase) || property.Equals(LabelIndexProperty, StringComparison.OrdinalIgnoreCase) || property.StartsWith(LabelPropertyPrefixDefault, StringComparison.OrdinalIgnoreCase); } } } vowpal-wabbit-8.5.0.dfsg1/cs/common/Serializer/Schema.cs000066400000000000000000000017371323713302100230450ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System.Collections.Generic; namespace VW.Serializer { /// /// Describes the serializatoin for a give type. /// public sealed class Schema { /// /// List of features to extract from type. /// public List Features { get; set; } /// /// Expression to access the label. /// public LabelExpression Label { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/common/Serializer/StringProcessing.cs000066400000000000000000000020761323713302100251450ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- namespace VW.Serializer { /// /// Options for string pre-processing before feeding to VW native. /// public enum StringProcessing { /// /// Spaces are replaced with underscores. /// Escape, /// /// Strings are split on space, producing individual features. /// Split, /// /// Spaces are replaced with underscores and the property name is used as a prefix. /// EscapeAndIncludeName } } vowpal-wabbit-8.5.0.dfsg1/cs/common/VowpalWabbitDecisionServiceInteractionHeader.cs000066400000000000000000000025711323713302100304420ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- namespace VW { /// /// Decision Service interaction header information. /// public sealed class VowpalWabbitDecisionServiceInteractionHeader { /// /// EventId extracted from Decision Service Interaction JSON data. /// public string EventId { get; set; } /// /// Actions extracted from Decision Service Interaction JSON data. /// public int[] Actions { get; set; } /// /// Probabilities extracted from Decision Service Interaction JSON data. /// public float[] Probabilities { get; set; } /// /// Probability of drop extracted from Decision Service Interaction JSON data. /// public float ProbabilityOfDrop { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/common/packages.config000066400000000000000000000002521323713302100221410ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/common/vw_common.csproj000066400000000000000000000214751323713302100224340ustar00rootroot00000000000000 Debug AnyCPU {E621E022-C1F8-433F-905A-AB9A3DE072B7} Library Properties VW VowpalWabbit.Common v4.5.2 512 0 true ..\vw_key.snk true $(SolutionDir)\x64\Debug\ DEBUG;TRACE full x64 prompt MinimumRecommendedRules.ruleset True False True False False True True True True True True True True True False True False True False False False False True False True True True False False True False False True Full %28none%29 0 ..\..\vowpalwabbit\x64\Debug\VowpalWabbit.Common.XML $(SolutionDir)\x64\Release\ TRACE true pdbonly x64 prompt MinimumRecommendedRules.ruleset ..\..\vowpalwabbit\x64\Release\VowpalWabbit.Common.XML true ..\..\vowpalwabbit\x64\DebugLeakCheck\ DEBUG;TRACE full x64 prompt MinimumRecommendedRules.ruleset This project references NuGet package(s) that are missing on this computer. Use NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. vowpal-wabbit-8.5.0.dfsg1/cs/cs/000077500000000000000000000000001323713302100163125ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/cs/ActionDependentFeature.cs000066400000000000000000000024721323713302100232260ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- namespace VW { /// /// A tuple of an action dependent feature and the corresponding index. /// /// The action dependent feature type. public sealed class ActionDependentFeature { internal ActionDependentFeature(int index, TActionDependentFeature feature) { this.Index = index; this.Feature = feature; } /// /// The index within the multi-line example. /// public int Index { get; private set; } /// /// The index within the multi-line example. /// public float Probability { get; set; } /// /// The feature object. /// public TActionDependentFeature Feature { get; private set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Extensions.cs000066400000000000000000000046071323713302100210070ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Diagnostics.Contracts; using System.Linq; using System.Text; using System.Threading.Tasks; namespace VW { /// /// LINQ extensions. /// public static class Extensions { /// /// Returns the elements specified by indicies/ /// /// The element type. /// The enumerable source. /// The indicies to be selected. /// The subset of elements. public static T[] Subset(this IEnumerable source, int[] indicies) { Contract.Requires(source != null); Contract.Requires(indicies != null); // re-shuffle var result = new T[indicies.Length]; var i = 0; foreach (var item in source) { result[indicies[i]] = item; i++; } return result; } /// /// Returns the index of the first element matching . /// /// The collection type. /// The source enumerable. /// The predicate to match. /// The index of the first element to match or -1 if none matched. public static int IndexOf(this IEnumerable source, Predicate predicate) { Contract.Requires(source != null); Contract.Requires(predicate != null); var i = 0; foreach (var t in source) { if (predicate(t)) { return i; } i++; } return -1; } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/ObjectFactory.cs000066400000000000000000000060211323713302100213760ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; namespace VW { /// /// Helper to conveniently create . /// public static class ObjectFactory { /// /// Disposable object factory. /// /// The disposable context needed to create objects of . /// The type of the objects to be created. public static ObjectFactory Create(TSource context, Func creator) where TSource : class, IDisposable { return new ObjectFactory(context, creator); } } /// /// Disposable object factory. /// /// The disposable context needed to create objects of . /// The type of the objects to be created. public class ObjectFactory : IDisposable where TSource : class, IDisposable { /// /// Factory function to create new instances. /// private readonly Func creator; /// /// The source object passed to . /// private TSource source; /// /// True if this instance is already disposed. /// private bool disposed; internal ObjectFactory(TSource source, Func creator) { this.source = source; this.creator = creator; this.disposed = false; } /// /// Creates a new object of type T. /// public TObject Create() { return this.creator(source); } /// /// Cleanup. /// public void Dispose() { this.Dispose(true); GC.SuppressFinalize(this); } private void Dispose(bool disposing) { if (disposing) { if (!disposed) { if (this.source != null) { this.source.Dispose(); this.source = null; } this.disposed = true; } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/ObjectPool.cs000066400000000000000000000177261323713302100207160ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Diagnostics.CodeAnalysis; using System.Diagnostics.Contracts; using System.Threading; namespace VW { /// /// Thread-safe object pool supporting versioned updates. /// /// The disposable context needed to create objects of . /// The type of the objects to be created. public class ObjectPool : IDisposable where TSource : class, IDisposable where TObject : class, IDisposable { /// /// Lock resources /// private readonly ReaderWriterLockSlim rwLockSlim; /// /// Version of the factory function. /// private int version; /// /// Used to create new pooled objects. /// private ObjectFactory factory; /// /// The actual pool. /// /// /// To maximize reuse of previously cached items within the pooled objects. /// (e.g. cached action dependent features) /// private Stack> pool; /// /// Initializes a new ObjectPool. /// /// /// An optional factory to create pooled objects on demand. /// will throw if the factory is still null when called. /// public ObjectPool(ObjectFactory factory = null) { this.rwLockSlim = new ReaderWriterLockSlim(); this.pool = new Stack>(); this.factory = factory; } /// /// Updates the object factory in a thread-safe manner. /// /// The new object factory to be used. public void UpdateFactory(ObjectFactory factory) { Stack> oldPool; ObjectFactory oldFactory; this.rwLockSlim.EnterWriteLock(); try { if (this.pool == null) { throw new ObjectDisposedException("ObjectPool already disposed"); } this.version++; oldFactory = this.factory; this.factory = factory; oldPool = this.pool; this.pool = new Stack>(); } finally { this.rwLockSlim.ExitWriteLock(); } // dispose outdated items foreach (var item in oldPool) { item.Value.Dispose(); } // dispose factory if (oldFactory != null) { oldFactory.Dispose(); } } /// /// Returns an instance of TObject from the pool or creates a new instance using the objectFactory /// if the pool is empty. /// /// This method is thread-safe. public PooledObject GetOrCreate() { int localVersion; ObjectFactory localFactory; this.rwLockSlim.EnterUpgradeableReadLock(); try { if (this.pool == null) { throw new ObjectDisposedException("ObjectPool already disposed"); } if (this.pool.Count == 0) { // create a consistent copy localVersion = this.version; localFactory = this.factory; } else { this.rwLockSlim.EnterWriteLock(); try { if (this.pool == null) { throw new ObjectDisposedException("ObjectPool already disposed"); } return this.pool.Pop(); } finally { this.rwLockSlim.ExitWriteLock(); } } } finally { this.rwLockSlim.ExitUpgradeableReadLock(); } if (localFactory == null) { throw new InvalidOperationException("Factory must be initialized before calling Get()"); } // invoke the factory outside of the lock return new PooledObject(this, localVersion, localFactory.Create()); } /// /// Returns to the pool of objects, given the version is still the same. /// Otherwise is disposed. /// /// The object to be returned. internal void ReturnObject(PooledObject pooledObject) { Contract.Ensures(pooledObject != null); this.rwLockSlim.EnterUpgradeableReadLock(); try { if (this.version == pooledObject.Version && this.pool != null) { this.rwLockSlim.EnterWriteLock(); try { // double check if (this.version == pooledObject.Version && this.pool != null) { // it's the same version, return to pool this.pool.Push(pooledObject); return; } } finally { this.rwLockSlim.ExitWriteLock(); } } } finally { this.rwLockSlim.ExitUpgradeableReadLock(); } // outdated pooledObject.Value.Dispose(); } /// /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources. /// public void Dispose() { this.Dispose(true); GC.SuppressFinalize(this); } private void Dispose(bool disposing) { if (disposing) { this.rwLockSlim.EnterWriteLock(); try { // Dispose pool items if (this.pool != null) { foreach (var item in this.pool) { item.Value.Dispose(); } this.pool = null; } // Dispose factory if (this.factory != null) { this.factory.Dispose(); this.factory = null; } } finally { this.rwLockSlim.ExitWriteLock(); } } } } }vowpal-wabbit-8.5.0.dfsg1/cs/cs/PooledObject.cs000066400000000000000000000042571323713302100212220ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; namespace VW { /// /// A strongly-typed pooled object. /// /// The disposable context needed to create objects of . /// The type of the objects to be created. public sealed class PooledObject : IDisposable where TSource : class, IDisposable where TObject : class, IDisposable { /// /// The parent pool. /// private readonly ObjectPool pool; /// /// Initializes a new instance of the class. /// /// The parent pool. /// The version of the pool at time of creation of this instance. /// The actual pooled object. internal PooledObject(ObjectPool pool, int version, TObject value) { this.pool = pool; this.Value = value; this.Version = version; } /// /// The actual value. /// public TObject Value { get; private set; } /// /// Factory version used to create Value. /// internal int Version { get; private set; } /// /// Return to pool. /// public void Dispose() { // don't keep empty objects in pool if (this.Value != null) this.pool.ReturnObject(this); } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Properties/000077500000000000000000000000001323713302100204465ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/cs/Properties/AssemblyInfo.cs000066400000000000000000000022131323713302100233660ustar00rootroot00000000000000//------------------------------------------------------------------------------ // // This code was generated by a tool. // Runtime Version:4.0.30319.42000 // // Changes to this file may cause incorrect behavior and will be lost if // the code is regenerated. // //------------------------------------------------------------------------------ [assembly: System.Reflection.AssemblyTitle("Vowpal Wabbit")] [assembly: System.Reflection.AssemblyDescription("Vowpal Wabbit")] [assembly: System.Reflection.AssemblyCompany("Microsoft Corp")] [assembly: System.Reflection.AssemblyProduct("Vowpal Wabbit")] [assembly: System.Reflection.AssemblyCopyright("Copyright (C) Microsoft Corp 2012-2016, Yahoo! Inc. 2007-2012, and many individua" + "l contributors. All rights reserved")] [assembly: System.Runtime.InteropServices.ComVisible(false)] [assembly: System.CLSCompliant(false)] [assembly: System.Runtime.InteropServices.Guid("6a577997-af00-4ca0-8453-fdc8bbdf2a57")] [assembly: System.Reflection.AssemblyVersion("8.4.0.1")] [assembly: System.Reflection.AssemblyFileVersion("8.4.0.1")] vowpal-wabbit-8.5.0.dfsg1/cs/cs/Properties/AssemblyInfoInternal.cs000066400000000000000000000010471323713302100250670ustar00rootroot00000000000000using System.Reflection; using System.Runtime.CompilerServices; using System.Runtime.InteropServices; // make internals available to unit test [assembly: InternalsVisibleTo("cs_unittest,PublicKey=" + "0024000004800000940000000602000000240000525341310004000001000100515aa9bda65291" + "811af92b381378bd271aff3a9e177bac69ff0e85874952fd82c0fbcb53f4e968181d07418481ee" + "2be97522d44c324aa5c683dafaa449fe66ddc65e1d9b3c0600c8820bd2be6401c6888ea88864ef" + "0b6ae5bfbf450aa1f548568d638913d82954195947e394c225cca2cd2f8132d525c2fdc0c57835" + "b87200aa")] vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/000077500000000000000000000000001323713302100204235ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/AnnotationInspector.cs000066400000000000000000000170011323713302100247520ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Diagnostics.Contracts; using System.Linq; using System.Linq.Expressions; using System.Reflection; using VW.Labels; using VW.Serializer.Attributes; using VW.Serializer.Intermediate; namespace VW.Serializer { /// /// Utility class analyzing compile-time annotation. /// public static class TypeInspector { /// /// All properties are used as features. /// public static readonly ITypeInspector All; /// /// Only properties annotated using Feature attribute are considered. /// public static readonly ITypeInspector Default; static TypeInspector() { All = new AnnotationInspectorAll(); Default = new AnnotationInspectorDefault(); } private sealed class AnnotationInspectorDefault : ITypeInspector { public Schema CreateSchema(VowpalWabbitSettings settings, Type type) { return TypeInspector.CreateSchema(type, featurePropertyPredicate: (_, attr) => attr != null, labelPropertyPredicate: (_, attr) => attr != null); } } private sealed class AnnotationInspectorAll : ITypeInspector { public Schema CreateSchema(VowpalWabbitSettings settings, Type type) { return TypeInspector.CreateSchema(type, featurePropertyPredicate: (_, __) => true, labelPropertyPredicate: (_, __) => true); } } private static Schema CreateSchema(Type type, Func featurePropertyPredicate, Func labelPropertyPredicate) { Contract.Requires(type != null); Contract.Requires(featurePropertyPredicate != null); Contract.Requires(labelPropertyPredicate != null); var validExpressions = new Stack>(); // CODE example != null validExpressions.Push(valueExpression => Expression.NotEqual(valueExpression, Expression.Constant(null))); return CreateSchema( null, type, null, null, null, // CODE example valueExpression => valueExpression, validExpressions, featurePropertyPredicate, labelPropertyPredicate); } private static Schema CreateSchema( FeatureExpression parent, Type type, string parentNamespace, char? parentFeatureGroup, bool? parentDictify, Func valueExpressionFactory, Stack> valueValidExpressionFactories, Func featurePropertyPredicate, Func labelPropertyPredicate) { var props = type.GetProperties(BindingFlags.Instance | BindingFlags.GetProperty | BindingFlags.Public); var localFeatures = (from p in props let declaredAttr = (FeatureAttribute)p.GetCustomAttributes(typeof(FeatureAttribute), true).FirstOrDefault() where featurePropertyPredicate(p, declaredAttr) let attr = declaredAttr ?? new FeatureAttribute() select new FeatureExpression( featureType: p.PropertyType, name: attr.Name ?? p.Name, // CODE example.Property valueExpressionFactory: valueExpression => Expression.Property(valueExpressionFactory(valueExpression), p), // @Reverse: make sure conditions are specified in the right order valueValidExpressionFactories: valueValidExpressionFactories.Reverse().ToList(), @namespace: attr.Namespace ?? parentNamespace, featureGroup: attr.InternalFeatureGroup ?? parentFeatureGroup, enumerize: attr.Enumerize, variableName: p.Name, order: attr.Order, addAnchor: attr.AddAnchor, stringProcessing: attr.StringProcessing, dictify: attr.InternalDictify ?? parentDictify, parent: parent) ).ToList(); var localLabels = from p in props let declaredAttr = (LabelAttribute)p.GetCustomAttributes(typeof(LabelAttribute), true).FirstOrDefault() where labelPropertyPredicate(p, declaredAttr) || typeof(ILabel).IsAssignableFrom(p.PropertyType) let attr = declaredAttr ?? new LabelAttribute() let labelType = p.PropertyType where typeof(ILabel).IsAssignableFrom(labelType) || p.PropertyType == typeof(string) select new LabelExpression { Name = p.Name, LabelType = p.PropertyType, // CODE example.Property ValueExpressionFactory = valueExpression => Expression.Property(valueExpressionFactory(valueExpression), p), // @Reverse: make sure conditions are specified in the right order ValueValidExpressionFactories = valueValidExpressionFactories.Reverse().ToList() }; // Recurse var schemas = localFeatures .Select(f => { // CODE example.Prop1.Prop2 != null valueValidExpressionFactories.Push(valueExpression => Expression.NotEqual(f.ValueExpressionFactory(valueExpression), Expression.Constant(null))); var subSchema = CreateSchema(f, f.FeatureType, f.Namespace, f.FeatureGroup, f.Dictify, f.ValueExpressionFactory, valueValidExpressionFactories, featurePropertyPredicate, labelPropertyPredicate); valueValidExpressionFactories.Pop(); return subSchema; }) .ToList(); return new Schema { Features = localFeatures.Union(schemas.SelectMany(s => s.Features)).ToList(), Label = localLabels.Union(schemas.Select(s => s.Label)).FirstOrDefault(l => l != null) }; } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/Attributes/000077500000000000000000000000001323713302100225515ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/Attributes/CacheableAttribute.cs000066400000000000000000000020471323713302100266160ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; namespace VW.Serializer.Attributes { /// /// Annotate an example class that's generated output should be cached. /// [AttributeUsage(AttributeTargets.Class)] public sealed class CacheableAttribute : Attribute { /// /// Specify an equality comparer to be used for the dictionary cache. /// If non is specified, default behavior of the /// public Type EqualityComparer { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/Attributes/FeatureAttribute.cs000066400000000000000000000072311323713302100263620ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using VW.Serializer; namespace VW.Serializer.Attributes { /// /// Annotate properties that should be serialized to Vowpal Wabbit /// [AttributeUsage(AttributeTargets.Property)] public sealed class FeatureAttribute : Attribute { /// /// Initializes a new instance of the class. /// public FeatureAttribute() { this.Enumerize = false; this.AddAnchor = false; this.StringProcessing = StringProcessing.Split; } /// /// The namespace. Corresponds to the string literal after | in the native VW line format.
/// e.g. user in "|user :0.1 :0.2" /// public string Namespace { get; set; } /// /// If true, features will be converted to string and then hashed. /// In VW line format: Age:15 (Enumerize=false), Age_15 (Enumerize=true) /// /// Defaults to false. public bool Enumerize { get; set; } /// /// If true, an anchoring feature (0:1) will be inserted at front. /// This is required if --interact is used to mark the beginning of the feature namespace, /// as 0-valued features are removed. /// /// Defaults to false. public bool AddAnchor { get; set; } /// /// If true, the string serialization will collect the feature into a dictionary and output a surrogate. /// /// Defaults to null, which inherits from parent. If no parent information available, defaults to false. public bool Dictify { get { return InternalDictify ?? false; } set { this.InternalDictify = value; } } /// /// Cope with potential null values. /// internal bool? InternalDictify { get; set; } /// /// Cope with potential null values. /// internal char? InternalFeatureGroup { get; set; } /// /// The regular VW string interface interprets the first character of the namespace as the feature group. /// public char FeatureGroup { get { return InternalFeatureGroup ?? VowpalWabbitConstants.DefaultNamespace; } set { this.InternalFeatureGroup = value; } } /// /// Allows feature name override. /// /// Defaults to reflected property name. public string Name { get; set; } /// /// Specify the serialization order. /// public int Order { get; set; } /// /// Configures string pre-processing before hashing. All options are compatible with VW string format. /// /// Defaults to public StringProcessing StringProcessing { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/Attributes/LabelAttribute.cs000066400000000000000000000013611323713302100260040ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; namespace VW.Serializer.Attributes { /// /// Used to annotate properties designated as labels. /// [AttributeUsage(AttributeTargets.Property)] public sealed class LabelAttribute : Attribute { } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/ConfigInspector.cs000066400000000000000000000204541323713302100240530ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Linq; using System.Linq.Expressions; using System.Reflection; using System.Text.RegularExpressions; namespace VW.Serializer { /// /// Constructs a marshalling schema based on string based configuration using /// this format: Property.Property.Property(Enumerize = true, Namespace = , ...) /// public static class ConfigInspector { /// /// Format: Property.Property.Property(Enumerize = true, Namespace = , ...) /// private static readonly Regex LineRegex = new Regex(@" ^\s* (?[^\s.\(]+) # first property (?:\. (?[^\s.\(]+) )* # other properties (\( # first attribute (?\w+) \s* = \s* (?[^,)\s]+) (?:\s*,\s* # other attributes (?\w+) \s* = \s* (?[^,)\s]+) \s* )* \))? \s*$ ", RegexOptions.IgnorePatternWhitespace | RegexOptions.Compiled); /// /// Creates a for the given . /// /// The type to be inspected. /// An optional config restricting the properties to be used as features. /// An otpional callback to signal warnings during inspection. /// Returns a schema describing the feature marshalling. public static Schema CreateSchema(Type type, string config, Action warnings = null) { if (warnings == null) warnings = msg => { }; var featureExpressionCtor = typeof(FeatureExpression).GetConstructors()[0]; var featureExpressions = new List(); var lines = config.Split(new[] { "\r\n", "\n" }, StringSplitOptions.RemoveEmptyEntries) .Select(l => l.Trim()) .Where(l => l.Length > 0); foreach (var line in lines) { var match = LineRegex.Match(line); if (!match.Success) { warnings(string.Format("Unable to parse: '{0}'", line)); continue; } var propPath = GetGroupMatch(match.Groups["prop"]) .ToList(); var names = GetGroupMatch(match.Groups["attrname"]) .Select(n => n.ToLowerInvariant()) .ToList(); var values = GetGroupMatch(match.Groups["attrvalue"]) .ToList(); if (names.Count != values.Count) { warnings(string.Format("Number of attribute names ({0}) and values ({1}) need match. '{2}'", names.Count, values.Count, line)); continue; } var currentType = type; bool valid = true; var properties = new List(); foreach (var prop in propPath) { var property = currentType.GetProperty(prop); // ignore, since we can't find if (property == null) { warnings(string.Format("Property '{0}' not found in type: '{1}'. Specification: {2}", prop, currentType, line)); valid = false; break; } properties.Add(property); currentType = property.PropertyType; } if (!valid || properties.Count == 0) { warnings(string.Format("No properties found on '{0}' specified by '{1}'", type, line)); continue; } Func valueExpression = e => WalkPropertyPath(e, properties); var valueValidExpression = new List>(); for (int i = 1; i < properties.Count - 1; i++) { valueValidExpression.Add(e => Expression.NotEqual( WalkPropertyPath(e, properties.Take(i)), Expression.Constant(null))); } var arguments = featureExpressionCtor.GetParameters() .Select(p => { var index = names.IndexOf(p.Name.ToLowerInvariant()); if (index == -1) return p.DefaultValue; var value = values[index]; if (p.ParameterType == typeof(string)) return ParseString(value); if (p.ParameterType == typeof(char?)) return ParseChar(value); if (p.ParameterType == typeof(bool)) return Convert.ToBoolean(value); if (p.ParameterType == typeof(int?)) return Convert.ToInt32(value); warnings(string.Format("Unsupported type '{0}' for parameter '{1}'", p.ParameterType, p.Name)); return p.DefaultValue; }) .Skip(4) .ToList(); // name, valueExpressionFactory, valueValidExpressionFactories arguments.InsertRange(0, new object[] { properties.Last().PropertyType, properties.Last().Name, valueExpression, valueValidExpression }); featureExpressions.Add((FeatureExpression)featureExpressionCtor.Invoke(arguments.ToArray())); } // TODO: adf? // p1.p2(Multi) // have a different return type... // run twice. separate schemas for shared and ADF // // Options from enum to factories // new VowpalWabbit return new Schema { Features = featureExpressions }; } private static string ParseString(string value) { // supports "Markus", Markus, "Markus, Markus" var match = Regex.Match(value.Trim(), "^\"?([^\"]+)\"?$"); if (!match.Success) throw new ArgumentException("Unable to parse: '" + value + "'"); return match.Groups[1].Value; } private static char ParseChar(string value) { var match = Regex.Match(value.Trim(), "^\'?([^'])'?$"); if (!match.Success) throw new ArgumentException("Unable to parse: '" + value + "'"); return match.Groups[1].Value[0]; } private static Expression WalkPropertyPath(Expression e, IEnumerable properties) { var propAccess = Expression.Property(e, properties.First()); var leftProperties = properties.Skip(1); if (leftProperties.Count() == 0) return propAccess; return WalkPropertyPath(propAccess, leftProperties); } private static IEnumerable GetGroupMatch(Group group) { if (group.Success) { for (int i = 0; i < group.Captures.Count; i++) yield return group.Captures[i].Value; } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/IVowpalWabbitConverter.cs000066400000000000000000000020301323713302100253470ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using VW.Serializer.Intermediate; namespace VW.Serializer { /// /// Types implementing custom serialization to VW should implement this interface. /// public interface IVowpalWabbitSerializable { /// /// Marshals this object into native VW /// /// /// /// void Marshal(VowpalWabbitMarshalContext ctx, Namespace ns, Feature feature); } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/IVowpalWabbitMultiExampleSerializerCompiler.cs000066400000000000000000000014131323713302100315370ustar00rootroot00000000000000using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; namespace VW.Serializer { /// /// Interface for multi-example serializer compilers. /// /// The example type. public interface IVowpalWabbitMultiExampleSerializerCompiler { /// /// Returns the number of action dependent examples holds. /// /// The example to inspect. /// Returns the number of action dependent examples holds. int GetNumberOfActionDependentExamples(TExample example); } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/IVowpalWabbitSerializer.cs000066400000000000000000000051531323713302100255220ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using VW.Labels; namespace VW.Serializer { /// /// Abstraction for single vs. multiline examples. /// /// The user-defined type to be serialized. public interface IVowpalWabbitSerializer : IDisposable { /// /// True if Vowpal Wabbit strings are generated, false otherwise. /// bool EnableStringExampleGeneration { get; } /// /// True if this serializer caches examples, false otherwise. /// bool CachesExamples { get; } /// /// Serializes the given into a Vowpal Wabbit string. /// /// The example to serialize. /// The optional label to serialize. /// The optional index of the example, the should be attributed to. /// Dictionary used to collect dictifyed features. /// Dictionary used to collect dictifyed features. /// The serialized Vowpal Wabbit string formatted example. string SerializeToString(TExample example, ILabel label = null, int? index = null, Dictionary dictionary = null, Dictionary fastDictionary = null); /// /// Serializes the given into a native Vowpal Wabbit example. /// /// The example to seralize. /// The optional label to serialize. /// The optional index of the example, the should be attributed to. /// A Vowpal Wabbit example ready to be used for prediction and learning. VowpalWabbitExampleCollection Serialize(TExample example, ILabel label = null, int? index = null); } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/IVowpalWabbitSerializerCompiler.cs000066400000000000000000000021241323713302100272100ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- namespace VW.Serializer { /// /// Abstraction for single vs. multiline examples. /// /// The user-defined type to be serialized. public interface IVowpalWabbitSerializerCompiler { /// /// Creates a new serializer for the given type. /// /// The VW instance this serializer is associated with. /// A ready to use serializer. IVowpalWabbitSerializer Create(VowpalWabbit vw); } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/Intermediate/000077500000000000000000000000001323713302100230355ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/Intermediate/EnumerizedFeature.cs000066400000000000000000000057531323713302100270210ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; namespace VW.Serializer.Intermediate { /// /// Feature description for enumerized features. Instead of estimating a single parameter/weight /// for a given integer, VW will estimate a parameter/weight for each value (one-hot encoding, dummy variables) /// /// The value type. public sealed class EnumerizedFeature : Feature { private readonly VowpalWabbit vw; private readonly Namespace ns; /// /// Enum hashing function. /// private readonly Func enumHashing; /// /// Initializes a new EnumerizedFeature. /// /// Vowpal Wabbit instance required for hashing. /// The associated namespace. /// The feature name. /// /// If true, enable dictionary extraction. /// The enumHash cache factory. public EnumerizedFeature(VowpalWabbit vw, Namespace ns, string name, bool addAnchor, bool dictify, Func, Func> enumHashing) : base(name, addAnchor, dictify) { if (!typeof(T).IsEnum) { throw new ArgumentException(string.Format("Type {0} must be enum", typeof(T))); } this.vw = vw; this.ns = ns; // initialize the enumHashing function this.enumHashing = enumHashing(this); } /// /// Hashes potentially using a cache. /// /// The value to be hashed. /// The hash of + public ulong FeatureHash(T value) { return this.enumHashing(value); } /// /// Hashes directly (no caching). /// /// The value to be hashed. /// The hash of + public ulong FeatureHashInternal(T value) { return this.vw.HashFeature( this.Name + Enum.GetName(typeof(T), value), this.ns.NamespaceHash); } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/Intermediate/Feature.cs000066400000000000000000000041111323713302100247540ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Diagnostics; using System.Diagnostics.Contracts; using System.Linq.Expressions; namespace VW.Serializer.Intermediate { /// /// The base feature description. /// [DebuggerDisplay("Feature({Name}, addAnchor: {AddAnchor}, dictify: {Dictify}")] public class Feature { /// /// Initializes a new Feature. /// /// /// /// public Feature(string name, bool addAnchor = false, bool dictify = false) { this.Name = name; this.AddAnchor = addAnchor; this.Dictify = dictify; } /// /// The origin property name is used as the feature name. /// public string Name { get; private set; } /// /// If true, an anchoring feature (0:1) will be inserted at front. /// This is required if --interact is used to mark the beginning of the feature namespace, /// as 0-valued features are removed. /// /// Defaults to false. public bool AddAnchor { get; private set; } /// /// If true, the string serialization will collect the feature into a dictionary and output a surrogate. /// /// Defaults to null, which inherits from parent. If no parent information available, defaults to false. public bool Dictify { get; private set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/Intermediate/Namespace.cs000066400000000000000000000063711323713302100252670ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Globalization; namespace VW.Serializer.Intermediate { /// /// The intermediate namespace representation. /// public sealed class Namespace { /// /// Initializes a new instance. /// /// VopwpalWabbit instance used for hashing. /// The namespace name. /// Defaults to space, if null. public Namespace(VowpalWabbit vw, string name, char? featureGroup) { this.Name = name; this.FeatureGroup = featureGroup ?? VowpalWabbitConstants.DefaultNamespace; if (featureGroup == null && !string.IsNullOrWhiteSpace(name)) { throw new ArgumentException("If Namespace is provided, FeatureGroup must be set too"); } // compute shared namespace hash this.NamespaceHash = name == null ? vw.HashSpace(this.FeatureGroup.ToString()) : vw.HashSpace(this.FeatureGroup + this.Name); if (vw.Settings.EnableStringExampleGeneration) this.NamespaceString = string.Format( CultureInfo.InvariantCulture, " |{0}{1}", this.FeatureGroup, this.Name); } /// /// Initializes a new instance. /// /// VopwpalWabbit instance used for hashing. /// The namespace name. First character is treated as feature group. Defaults to space. public Namespace(VowpalWabbit vw, string name = null) { if (string.IsNullOrWhiteSpace(name)) name = VowpalWabbitConstants.DefaultNamespace.ToString(); if (name.Length > 1) this.Name = name.Substring(1); this.FeatureGroup = name[0]; this.NamespaceHash = vw.HashSpace(name); if (vw.Settings.EnableStringExampleGeneration) this.NamespaceString = " |" + name; } /// /// Gets or sets the namespace name. /// public string Name { get; private set; } /// /// Gets or sets the feature group. /// public char FeatureGroup { get; private set; } /// /// The pre-calculated hash. /// public ulong NamespaceHash { get; private set; } /// /// The string representation of the namespace. /// public string NamespaceString { get; private set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/Intermediate/PreHashedFeature.cs000066400000000000000000000027761323713302100265570ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- namespace VW.Serializer.Intermediate { /// /// The intermediate feature representation. /// public sealed class PreHashedFeature : Feature { /// /// Initializes a new instance of the class. /// /// The vowpal wabbit instance. /// The namespace. /// The feature name/ /// True if an anchor needs to be added, false otherwise. /// public PreHashedFeature(VowpalWabbit vw, Namespace ns, string name, bool addAnchor = false, bool dictify = false) : base(name, addAnchor, dictify) { this.FeatureHash = vw.HashFeature(this.Name, ns.NamespaceHash); } /// /// The pre-hashed feature hash. /// public ulong FeatureHash { get; private set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/VowpalWabbitConstants.cs000066400000000000000000000014571323713302100252570ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- namespace VW.Serializer { /// /// String constants used by VW. /// public static class VowpalWabbitConstants { /// /// The VW default namespace is denoted by a blank. /// public const char DefaultNamespace = ' '; } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/VowpalWabbitDefaultMarshaller.cs000066400000000000000000000366441323713302100267100ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections; using System.Collections.Generic; using System.Diagnostics.Contracts; using System.Globalization; using System.Text.RegularExpressions; using VW.Labels; using VW.Serializer.Intermediate; namespace VW.Serializer { /// /// The default marshaller for most types supported by VW. /// public sealed partial class VowpalWabbitDefaultMarshaller { /// /// Singleton default marshaller as it is stateless. /// public static readonly VowpalWabbitDefaultMarshaller Instance = new VowpalWabbitDefaultMarshaller(); /// /// Marshals a boolean value into native VW. /// /// e.g. loggedIn = true yields "loggedIn" in VW native string format. /// e.g. loggedIn = false yields an empty string. /// /// The marshalling context. /// The namespace description. /// The feature description. /// The actual feature value. public void MarshalFeature(VowpalWabbitMarshalContext context, Namespace ns, PreHashedFeature feature, bool value) { Contract.Requires(context != null); Contract.Requires(ns != null); Contract.Requires(feature != null); if (!value) { return; } context.NamespaceBuilder.AddFeature(feature.FeatureHash, 1f); context.AppendStringExample(feature.Dictify, " {0}", feature.Name); } /// /// Marshals an enum value into native VW. /// /// The enum type. /// The marshalling context. /// The namespace description. /// The feature description. /// The actual feature value. /// Gender = Male yields "GenderMale" in VW native string format. public void MarshalEnumFeature(VowpalWabbitMarshalContext context, Namespace ns, EnumerizedFeature feature, T value) { Contract.Requires(context != null); Contract.Requires(ns != null); Contract.Requires(feature != null); context.NamespaceBuilder.AddFeature(feature.FeatureHash(value), 1f); context.AppendStringExample(feature.Dictify, " {0}{1}", feature.Name, value); } /// /// Marshals any type into native VW, by constructing a 1-hot encoding using . /// /// The type to be enumerized. /// The marshalling context. /// The namespace description. /// The feature description. /// The actual feature value. /// is , actual value '25' and is 'Age'. /// The result is equivalent to 'Age25'. /// public void MarshalEnumerizeFeature(VowpalWabbitMarshalContext context, Namespace ns, Feature feature, T value) { Contract.Requires(context != null); Contract.Requires(ns != null); Contract.Requires(feature != null); var stringValue = feature.Name + value.ToString(); context.NamespaceBuilder.AddFeature(context.VW.HashFeature(stringValue, ns.NamespaceHash), 1f); context.AppendStringExample(feature.Dictify, " {0}", stringValue); } private static Regex escapeCharacters = new Regex("[ \t|:]", RegexOptions.Compiled); /// /// Marshals the supplied string into VW native space. Spaces are escaped using '_'. /// Only is serialized, Name is ignored. /// /// The marshalling context. /// The namespace description. /// The feature description. /// The actual feature value. /// is "New York". Result is "New_York". public void MarshalFeatureStringEscape(VowpalWabbitMarshalContext context, Namespace ns, Feature feature, string value) { Contract.Requires(context != null); Contract.Requires(ns != null); Contract.Requires(feature != null); if (string.IsNullOrWhiteSpace(value)) return; // safe escape spaces value = escapeCharacters.Replace(value, "_"); var featureHash = context.VW.HashFeature(value, ns.NamespaceHash); context.NamespaceBuilder.AddFeature(featureHash, 1f); context.AppendStringExample(feature.Dictify, " {0}", value); } /// /// Marshals the supplied string into VW native space. Spaces are escaped using '_'. Includes the in the 1-hot encoded feature. /// /// The marshalling context. /// The namespace description. /// The feature description. /// The actual feature value. /// is "New York". Name is "Location". Result is "LocationNew_York". public void MarshalFeatureStringEscapeAndIncludeName(VowpalWabbitMarshalContext context, Namespace ns, Feature feature, string value) { Contract.Requires(context != null); Contract.Requires(ns != null); Contract.Requires(feature != null); if (string.IsNullOrWhiteSpace(value)) return; // safe escape spaces value = feature.Name + escapeCharacters.Replace(value, "_"); var featureHash = context.VW.HashFeature(value, ns.NamespaceHash); context.NamespaceBuilder.AddFeature(featureHash, 1f); context.AppendStringExample(feature.Dictify, " {0}", value); } /// /// Marshals the supplied string into VW native space, by splitting the word by white space. /// /// The marshalling context. /// The namespace description. /// The feature description. /// The actual feature value. /// is "New York". Result is "New York", corresponding to 2 featuers in VW native space. public void MarshalFeatureStringSplit(VowpalWabbitMarshalContext context, Namespace ns, Feature feature, string value) { if (string.IsNullOrWhiteSpace(value)) return; var words = value.Split((char[])null, StringSplitOptions.RemoveEmptyEntries); foreach (var s in words) { var featureHash = context.VW.HashFeature(escapeCharacters.Replace(s, "_"), ns.NamespaceHash); context.NamespaceBuilder.AddFeature(featureHash, 1f); } if (context.StringExample == null) { return; } foreach (var s in words) { context.AppendStringExample(feature.Dictify, " {0}", escapeCharacters.Replace(s, "_")); } } /// /// Transfers feature data to native space. /// /// /// /// The marshalling context. /// The namespace description. /// The feature description. /// The actual feature value. public void MarshalFeature(VowpalWabbitMarshalContext context, Namespace ns, Feature feature, IEnumerable> value) { Contract.Requires(context != null); Contract.Requires(ns != null); Contract.Requires(feature != null); if (value == null) { return; } foreach (var kvp in value) { context.NamespaceBuilder.AddFeature( context.VW.HashFeature(Convert.ToString(kvp.Key), ns.NamespaceHash), Convert.ToSingle(kvp.Value, CultureInfo.InvariantCulture)); } if (context.StringExample == null) { return; } foreach (var kvp in value) { context.AppendStringExample( feature.Dictify, " {0}:" + (context.VW.Settings.EnableStringFloatCompact ? "{1}" : "{1:E20}"), Convert.ToString(kvp.Key), Convert.ToSingle(kvp.Value, CultureInfo.InvariantCulture)); } } /// /// /// /// The marshalling context. /// The namespace description. /// The feature description. /// The actual feature value. public void MarshalFeature(VowpalWabbitMarshalContext context, Namespace ns, Feature feature, IDictionary value) { Contract.Requires(context != null); Contract.Requires(ns != null); Contract.Requires(feature != null); if (value == null) { return; } foreach (DictionaryEntry item in value) { context.NamespaceBuilder.AddFeature( context.VW.HashFeature(Convert.ToString(item.Key), ns.NamespaceHash), Convert.ToSingle(item.Value, CultureInfo.InvariantCulture)); } if (context.StringExample == null) { return; } foreach (DictionaryEntry item in value) { context.AppendStringExample( feature.Dictify, " {0}:" + (context.VW.Settings.EnableStringFloatCompact ? "{1}" : "{1:E20}"), Convert.ToString(item.Key), Convert.ToSingle(item.Value, CultureInfo.InvariantCulture)); } } /// /// /// /// The marshalling context. /// The namespace description. /// The feature description. /// The actual feature value. public void MarshalFeature(VowpalWabbitMarshalContext context, Namespace ns, Feature feature, IEnumerable value) { Contract.Requires(context != null); Contract.Requires(ns != null); Contract.Requires(feature != null); if (value == null) return; foreach (var item in value) context.NamespaceBuilder.AddFeature(context.VW.HashFeature(item.Replace(' ', '_'), ns.NamespaceHash), 1f); if (context.StringExample == null) return; foreach (var item in value) context.AppendStringExample(feature.Dictify, " {0}", item); } /// /// /// /// The marshalling context. /// The namespace description. /// The feature description. /// The actual feature value. public void MarshalFeature(VowpalWabbitMarshalContext context, Namespace ns, Feature feature, IVowpalWabbitSerializable value) { Contract.Requires(context != null); Contract.Requires(ns != null); Contract.Requires(feature != null); if (value == null) return; value.Marshal(context, ns, feature); } /// /// /// /// The marshalling context. /// The namespace description. /// public int MarshalNamespace(VowpalWabbitMarshalContext context, Namespace ns, Action featureVisits) { try { // the namespace is only added on dispose, to be able to check if at least a single feature has been added context.NamespaceBuilder = context.ExampleBuilder.AddNamespace(ns.FeatureGroup); var position = 0; var stringExample = context.StringExample; if (context.StringExample != null) position = stringExample.Append(ns.NamespaceString).Length; featureVisits(); if (context.StringExample != null) { if (position == stringExample.Length) // no features added, remove namespace stringExample.Length = position - ns.NamespaceString.Length; } return (int)context.NamespaceBuilder.FeatureCount; } finally { if (context.NamespaceBuilder != null) { context.NamespaceBuilder.Dispose(); context.NamespaceBuilder = null; } } } /// /// /// /// The marshalling context. /// public void MarshalLabel(VowpalWabbitMarshalContext context, ILabel label) { if (label == null) return; context.ExampleBuilder.ApplyLabel(label); // prefix with label if (context.StringExample != null) context.StringLabel = label.ToString(); } /// /// /// /// The marshalling context. /// public void MarshalLabel(VowpalWabbitMarshalContext context, string label) { if (label == null) return; context.ExampleBuilder.ApplyLabel(new StringLabel(label)); context.StringLabel = label; } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/VowpalWabbitDefaultMarshallerExt.tt000066400000000000000000000352401323713302100274020ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections; using System.Collections.Generic; using System.Diagnostics; using System.Diagnostics.Contracts; using System.Linq; using System.Globalization; using System.Text; using VW.Serializer.Intermediate; <#@ assembly name="System.Core" #> <#@ import namespace="System.Linq" #> <#@ output extension=".cs" #> namespace VW.Serializer { public partial class VowpalWabbitDefaultMarshaller { <# foreach(var t in new[] { typeof(byte), typeof(sbyte), typeof(Int16), typeof(Int32), typeof(UInt16), typeof(UInt32), typeof(float), typeof(Int64), typeof(UInt64), typeof(double) }) { #> /// /// Transfers feature data to native space. /// /// The marshalling context. /// The namespace description. /// The feature description. /// The feature value. public void MarshalFeature(VowpalWabbitMarshalContext context, Namespace ns, PreHashedFeature feature, <#=t#> value) { Contract.Requires(context != null); Contract.Requires(ns != null); Contract.Requires(feature != null); <# if(new[]{ typeof(Int64), typeof(UInt64), typeof(double) }.Contains(t)) {#> #if DEBUG if (value > float.MaxValue || value < float.MinValue) { Trace.TraceWarning("Precision lost for feature value: " + value); } #endif <# } #> context.NamespaceBuilder.AddFeature(feature.FeatureHash, (float)value); context.AppendStringExample( feature.Dictify, " {0}:" + (context.VW.Settings.EnableStringFloatCompact ? "{1}" : "{1:E20}"), feature.Name, value); } /// /// Transfers feature data to native space. /// /// The marshalling context. /// The namespace description. /// The feature description. /// The feature value. public unsafe void MarshalFeature(VowpalWabbitMarshalContext context, Namespace ns, Feature feature, <#=t#>[] value) { if (value == null) return; this.MarshalFeature(context, ns, feature, value, 0, value.Length); } /// /// Transfers feature data to native space. /// /// The marshalling context. /// The namespace description. /// The feature description. /// The feature values. /// Start offset for feature values. /// Length of feature values. public unsafe void MarshalFeature(VowpalWabbitMarshalContext context, Namespace ns, Feature feature, <#=t#>[] value, int offset, int length) { if (value == null) return; ulong i = 0; // support anchor feature if (feature.AddAnchor) { context.NamespaceBuilder.PreAllocate(value.Length + 1); context.NamespaceBuilder.AddFeature(ns.NamespaceHash, 1); i++; } else context.NamespaceBuilder.PreAllocate(value.Length); <# if (t == typeof(float)) { #> fixed (float* begin = value) { var temp = begin + offset; context.NamespaceBuilder.AddFeaturesUnchecked((ulong)(ns.NamespaceHash + i), temp, temp + length); } <# } else { #> for (var j = offset;j #if DEBUG if (v > float.MaxValue || v < float.MinValue) { Trace.TraceWarning("Precision lost for feature value: " + v); } #endif <# } #> context.NamespaceBuilder.AddFeature((ulong)(ns.NamespaceHash + i), (float)v); i++; } <# } #> if (context.StringExample == null) return; string featureString; if (feature.Dictify && context.FastDictionary != null) { if (context.FastDictionary.TryGetValue(value, out featureString)) { context.AppendStringExample(feature.Dictify, featureString); return; } } var featureBuilder = new StringBuilder(); // support anchor feature i = 0; if (feature.AddAnchor) { featureBuilder.Append(" 0:1"); i++; } var format = " {0}:" + (context.VW.Settings.EnableStringFloatCompact ? "{1}" : "{1:E20}"); for (var j = offset;j /// Transfers feature data to native space. /// /// The marshalling context. /// The namespace description. /// The feature description. /// The feature value. public void MarshalFeature(VowpalWabbitMarshalContext context, Namespace ns, Feature feature, IEnumerable<<#=t#>> value) { if (value == null) { return; } ulong i = 0; // support anchor feature if (feature.AddAnchor) { context.NamespaceBuilder.AddFeature(ns.NamespaceHash, 1); i++; } foreach (var v in value) { <# if(new[]{ typeof(Int64), typeof(UInt64), typeof(double) }.Contains(t)) {#> #if DEBUG if (v > float.MaxValue || v < float.MinValue) { Trace.TraceWarning("Precision lost for feature value: " + v); } #endif <# } #> context.NamespaceBuilder.AddFeature((ulong)(ns.NamespaceHash + i), (float)v); i++; } if (context.StringExample == null) return; string featureString; if (feature.Dictify && context.FastDictionary != null) { if (context.FastDictionary.TryGetValue(value, out featureString)) { context.AppendStringExample(feature.Dictify, featureString); return; } } var featureBuilder = new StringBuilder(); // support anchor feature i = 0; if (feature.AddAnchor) { featureBuilder.Append(" 0:1"); i++; } var format = " {0}:" + (context.VW.Settings.EnableStringFloatCompact ? "{1}" : "{1:E20}"); foreach (var v in value) { featureBuilder.AppendFormat( CultureInfo.InvariantCulture, format, i, v); i++; } featureString = featureBuilder.ToString(); if (feature.Dictify && context.FastDictionary != null) context.FastDictionary.Add(value, featureString); context.AppendStringExample(feature.Dictify, featureString); } <# } #> <# foreach(var t in new[] { typeof(byte), typeof(sbyte), typeof(Int16), typeof(Int32), typeof(UInt16) }) { #> <# foreach(var s in new[] { typeof(byte), typeof(sbyte), typeof(Int16), typeof(Int32), typeof(UInt16), typeof(UInt32), typeof(float), typeof(Int64), typeof(UInt64), typeof(double) }) { #> /// /// Transfers feature data to native space. /// /// The marshalling context. /// The namespace description. /// The feature description. /// The feature value. [ContractVerification(false)] public void MarshalFeature(VowpalWabbitMarshalContext context, Namespace ns, Feature feature, IDictionary<<#=t#>, <#=s#>> value) { Contract.Requires(context != null); Contract.Requires(ns != null); Contract.Requires(feature != null); if (value == null) return; foreach (var kvp in value) { <# if(new[]{ typeof(Int64), typeof(UInt64), typeof(double) }.Contains(s)) {#> #if DEBUG if (kvp.Value > float.MaxValue || kvp.Value < float.MinValue) { Trace.TraceWarning("Precision lost for feature value: " + kvp.Value); } #endif <# } #> context.NamespaceBuilder.AddFeature(ns.NamespaceHash + (ulong)kvp.Key, (float)kvp.Value); } if (context.StringExample == null) return; var format = " {0}:" + (context.VW.Settings.EnableStringFloatCompact ? "{1}" : "{1:E20}"); foreach (var kvp in value) { // TODO: not sure if negative numbers will work context.AppendStringExample( feature.Dictify, format, kvp.Key, kvp.Value); } } <# } #> <# } #> <# foreach(var s in new[] { typeof(byte), typeof(sbyte), typeof(Int16), typeof(Int32), typeof(UInt16), typeof(UInt32), typeof(float), typeof(Int64), typeof(UInt64), typeof(double) }) { #> /// /// Transfers feature data to native space. /// /// The marshalling context. /// The namespace description. /// The feature description. /// The feature value. [ContractVerification(false)] public void MarshalFeature(VowpalWabbitMarshalContext context, Namespace ns, Feature feature, IDictionary> value) { Contract.Requires(context != null); Contract.Requires(ns != null); Contract.Requires(feature != null); if (value == null) return; foreach (var kvp in value) { <# if(new[]{ typeof(Int64), typeof(UInt64), typeof(double) }.Contains(s)) {#> #if DEBUG if (kvp.Value > float.MaxValue || kvp.Value < float.MinValue) { Trace.TraceWarning("Precision lost for feature value: " + kvp.Value); } #endif <# } #> context.NamespaceBuilder.AddFeature( context.VW.HashFeature(kvp.Key, ns.NamespaceHash), (float)kvp.Value); } if (context.StringExample == null) return; var format = " {0}:" + (context.VW.Settings.EnableStringFloatCompact ? "{1}" : "{1:E20}"); foreach (var kvp in value) { // TODO: not sure if negative numbers will work context.AppendStringExample( feature.Dictify, format, kvp.Key, kvp.Value); } } /// /// Transfers feature data to native space. /// /// The marshalling context. /// The namespace description. /// The feature description. /// The feature value. [ContractVerification(false)] public int MarshalNamespace(VowpalWabbitMarshalContext context, Namespace ns, Feature feature, IDictionary[]> value) { Contract.Requires(context != null); Contract.Requires(ns != null); Contract.Requires(feature != null); if (value == null) return 0; int featureCount = 0; foreach (var kvp in value) { var perKeyNs = kvp.Key == null ? ns : new Namespace(context.VW, kvp.Key); try { // the namespace is only added on dispose, to be able to check if at least a single feature has been added context.NamespaceBuilder = context.ExampleBuilder.AddNamespace(perKeyNs.FeatureGroup); var position = 0; var stringExample = context.StringExample; if (context.StringExample != null) { position = stringExample.Append(perKeyNs.NamespaceString).Length; } this.MarshalFeature(context, perKeyNs, feature, kvp.Value); if (context.StringExample != null) { if (position == stringExample.Length) { // no features added, remove namespace stringExample.Length = position - ns.NamespaceString.Length; } } featureCount += (int)context.NamespaceBuilder.FeatureCount; } finally { if (context.NamespaceBuilder != null) { context.NamespaceBuilder.Dispose(); context.NamespaceBuilder = null; } } } return featureCount; } <# } #> } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/VowpalWabbitMarshalContext.cs000066400000000000000000000152321323713302100262330ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Diagnostics; using System.Globalization; using System.Linq; using System.Text; using System.Threading.Tasks; using VW; namespace VW.Serializer { /// /// Context containing state during example marshalling. /// [DebuggerDisplay("{GetHashCode()}: {ToString()}")] public class VowpalWabbitMarshalContext : IDisposable { /// /// If true disposes the example builder. Otherwise it's not owned by this instance. /// private bool disposeExampleBuilder = false; /// /// Initializes a new instance of the class. /// /// The VW instance the example will be imported to. /// Dictionary used for dictify operation. /// Dictionary used for dictify operation. public VowpalWabbitMarshalContext(IVowpalWabbitExamplePool vwPool, Dictionary dictionary = null, Dictionary fastDictionary = null) : this(vwPool.Native, new VowpalWabbitExampleBuilder(vwPool), dictionary, fastDictionary) { disposeExampleBuilder = true; } /// /// Initializes a new instance of the class. /// /// The VW instance the example will be imported to. /// A shared example builder. /// Dictionary used for dictify operation. /// Dictionary used for dictify operation. public VowpalWabbitMarshalContext(VowpalWabbit vw, VowpalWabbitExampleBuilder exampleBuilder, Dictionary dictionary = null, Dictionary fastDictionary = null) { this.VW = vw; this.ExampleBuilder = exampleBuilder; if (vw.Settings.EnableStringExampleGeneration) { this.StringExample = new StringBuilder(); this.Dictionary = dictionary; this.FastDictionary = fastDictionary; } } /// /// The VW instance the produce example will be imported to. /// public VowpalWabbit VW { get; private set; } /// /// See https://github.com/JohnLangford/vowpal_wabbit/wiki/Input-format for reference /// public StringBuilder StringExample { get; private set; } /// /// The VW string version of the label. /// public string StringLabel { get; set; } /// /// Creates the VW string for this example. /// /// public override string ToString() { if (this.StringExample == null) return null; var sb = new StringBuilder(); if (this.StringLabel != null) sb.Append(this.StringLabel); if (this.StringExample.Length > 0) { if (sb.Length > 0) sb.Append(' '); sb.Append(this.StringExample); } return sb.ToString(); } /// /// Used if dictify is true. Maps from serialized feature to surrogate key. /// public Dictionary Dictionary { get; private set; } /// /// Used if dictify is true. Maps from raw feature value (e.g. int[]) to serialized feature. /// public Dictionary FastDictionary { get; private set; } /// /// Used to build examples. /// public VowpalWabbitExampleBuilder ExampleBuilder { get; private set; } /// /// Used to build a namespace. /// public VowpalWabbitNamespaceBuilder NamespaceBuilder { get; set; } /// /// Formats based on to the string example buffer. /// /// If true, performs dictionarization on the serialized string and inserts a surrogate. /// The string format used to serialize . /// The arguments to the string format operation. public void AppendStringExample(bool dictify, string format, params object[] args) { if (this.StringExample != null) { var outputString = string.Format(CultureInfo.InvariantCulture, format, args); if (dictify && this.Dictionary != null) { string surrogate; if (!this.Dictionary.TryGetValue(outputString, out surrogate)) { // prefix to avoid number parsing surrogate = "d" + this.Dictionary.Count.ToString(CultureInfo.InvariantCulture); this.Dictionary.Add(outputString, surrogate); } this.StringExample.AppendFormat(" {0}", surrogate); } else { this.StringExample.Append(outputString); } } } /// /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources. /// public void Dispose() { this.Dispose(true); GC.SuppressFinalize(this); } private void Dispose(bool disposing) { if (disposing) { if (this.ExampleBuilder != null && disposeExampleBuilder) { this.ExampleBuilder.Dispose(); this.ExampleBuilder = null; } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/VowpalWabbitMultiExampleSerializer.cs000066400000000000000000000115571323713302100277450ustar00rootroot00000000000000using System; using System.Collections.Generic; using System.Diagnostics.Contracts; using System.Linq; using System.Text; using System.Threading.Tasks; using VW.Labels; namespace VW.Serializer { internal sealed class VowpalWabbitMultiExampleSerializer : IVowpalWabbitSerializer { private readonly VowpalWabbit vw; private readonly Func> adfAccessor; private VowpalWabbitSingleExampleSerializer sharedSerializer; private VowpalWabbitSingleExampleSerializer adfSerializer; internal VowpalWabbitMultiExampleSerializer(VowpalWabbit vw, VowpalWabbitSingleExampleSerializer sharedSerializer, VowpalWabbitSingleExampleSerializer adfSerializer, Func> adfAccessor) { // sharedSerializer can be null Contract.Requires(vw != null); Contract.Requires(adfSerializer != null); Contract.Requires(adfAccessor != null); this.vw = vw; this.sharedSerializer = sharedSerializer; this.adfSerializer = adfSerializer; this.adfAccessor = adfAccessor; } public bool EnableStringExampleGeneration { get { return this.vw.Settings.EnableStringExampleGeneration; } } public bool CachesExamples { get { return (this.sharedSerializer != null && this.sharedSerializer.CachesExamples) || this.adfSerializer.CachesExamples; } } public int GetNumberOfActionDependentExamples(TExample example) { var adfs = this.adfAccessor(example); return adfs == null ? 0 : adfs.Count(); } /// /// Serializes the given to VW string format. /// /// The example to serialize. /// The label to serialize. /// The optional index of the example, the should be attributed to. /// Dictionary used for dictify operation. /// Dictionary used for dictify operation. /// The resulting VW string. public string SerializeToString(TExample example, ILabel label = null, int? index = null, Dictionary dictionary = null, Dictionary fastDictionary = null) { var sb = new StringBuilder(); if (this.sharedSerializer != null) sb.AppendLine(this.sharedSerializer.SerializeToString(example, SharedLabel.Instance, null, dictionary, fastDictionary)); var adfCollection = this.adfAccessor(example); if (adfCollection != null) { var i = 0; foreach (var adfExample in adfCollection) { sb.AppendLine(this.adfSerializer.SerializeToString(adfExample, index != null && i == index ? label : null, null, dictionary, fastDictionary)); i++; } } return sb.ToString(); } public VowpalWabbitExampleCollection Serialize(TExample example, ILabel label = null, int? index = null) { VowpalWabbitExample shared = null; var adf = new List(); try { if (this.sharedSerializer != null) shared = this.sharedSerializer.Serialize(example, SharedLabel.Instance); var adfCollection = this.adfAccessor(example); if (adfCollection != null) { var i = 0; foreach (var adfExample in adfCollection) { adf.Add(this.adfSerializer.Serialize(adfExample, index != null && i == index ? label : null)); i++; } } return new VowpalWabbitMultiLineExampleCollection(this.vw, shared, adf.ToArray()); } catch (Exception) { if (shared != null) shared.Dispose(); foreach (var adfExample in adf) adfExample.Dispose(); throw; } } public void Dispose() { if (this.sharedSerializer != null) { this.sharedSerializer.Dispose(); this.sharedSerializer = null; } if (this.adfSerializer != null) { this.adfSerializer.Dispose(); this.adfSerializer = null; } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/VowpalWabbitMultiExampleSerializerCompiler.cs000066400000000000000000000126641323713302100314400ustar00rootroot00000000000000using System; using System.Collections.Generic; using System.Diagnostics.Contracts; using System.Linq; using System.Linq.Expressions; using System.Text; using System.Threading.Tasks; using VW.Reflection; namespace VW.Serializer { /// /// Factory class to create . /// public static class VowpalWabbitMultiExampleSerializerCompiler { /// /// Creates a serializer for based on and , /// /// The example type. /// Settings for inspection. /// The schema used for serializer creation. /// If the schema is valid a compiler is created, otherwise null. public static IVowpalWabbitSerializerCompiler TryCreate(VowpalWabbitSettings settings, Schema schema) { // check for _multi var multiFeature = schema.Features.FirstOrDefault(fe => fe.Name == settings.PropertyConfiguration.MultiProperty); if (multiFeature == null) return null; // multi example path // IEnumerable<> or Array var adfType = InspectionHelper.GetEnumerableElementType(multiFeature.FeatureType); if (adfType == null) throw new ArgumentException(settings.PropertyConfiguration.MultiProperty + " property must be array or IEnumerable<>. Actual type: " + multiFeature.FeatureType); var compilerType = typeof(VowpalWabbitMultiExampleSerializerCompilerImpl<,>).MakeGenericType(typeof(TExample), adfType); return (IVowpalWabbitSerializerCompiler)Activator.CreateInstance(compilerType, settings, schema, multiFeature); } private sealed class VowpalWabbitMultiExampleSerializerCompilerImpl : IVowpalWabbitSerializerCompiler, IVowpalWabbitMultiExampleSerializerCompiler { private readonly VowpalWabbitSingleExampleSerializerCompiler sharedSerializerCompiler; private readonly VowpalWabbitSingleExampleSerializerCompiler adfSerializerComputer; private readonly Func> adfAccessor; public VowpalWabbitMultiExampleSerializerCompilerImpl(VowpalWabbitSettings settings, Schema schema, FeatureExpression multiFeature) { Contract.Requires(settings != null); Contract.Requires(schema != null); Contract.Requires(multiFeature != null); var nonMultiFeatures = schema.Features.Where(fe => fe != multiFeature).ToList(); this.sharedSerializerCompiler = nonMultiFeatures.Count == 0 ? null : new VowpalWabbitSingleExampleSerializerCompiler( new Schema { Features = nonMultiFeatures }, settings == null ? null : settings.CustomFeaturizer, !settings.EnableStringExampleGeneration); this.adfSerializerComputer = new VowpalWabbitSingleExampleSerializerCompiler( settings.TypeInspector.CreateSchema(settings, typeof(TActionDependentFeature)), settings == null ? null : settings.CustomFeaturizer, !settings.EnableStringExampleGeneration); var exampleParameter = Expression.Parameter(typeof(TExample), "example"); // CODE condition1 && condition2 && condition3 ... var condition = multiFeature.ValueValidExpressionFactories .Skip(1) .Aggregate( multiFeature.ValueValidExpressionFactories.First()(exampleParameter), (cond, factory) => Expression.AndAlso(cond, factory(exampleParameter))); var multiExpression = multiFeature.ValueExpressionFactory(exampleParameter); // CODE example => (IEnumerable)(example._multi != null ? example._multi : null) var expr = Expression.Lambda>>( Expression.Condition( condition, multiExpression, Expression.Constant(null, multiExpression.Type), typeof(IEnumerable)), exampleParameter); this.adfAccessor = (Func>)expr.CompileToFunc(); } public int GetNumberOfActionDependentExamples(TExample example) { var adfs = this.adfAccessor(example); return adfs == null ? 0 : adfs.Count(); } public IVowpalWabbitSerializer Create(VowpalWabbit vw) { return new VowpalWabbitMultiExampleSerializer( vw, this.sharedSerializerCompiler != null ? this.sharedSerializerCompiler.Create(vw) : null, this.adfSerializerComputer.Create(vw), this.adfAccessor); } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/VowpalWabbitSerializerFactory.cs000066400000000000000000000124461323713302100267440ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Linq; using System.Linq.Expressions; using System.Reflection; using System.Reflection.Emit; using VW.Serializer.Attributes; using VW.Serializer.Intermediate; using VW.Labels; using System.IO; using System.Runtime.CompilerServices; using System.Diagnostics.Contracts; using VW.Reflection; using System.Collections.ObjectModel; namespace VW.Serializer { /// /// Factory to ease creation of serializers. /// public static class VowpalWabbitSerializerFactory { /// /// Example and example result type based serializer cache. /// private static readonly Dictionary SerializerCache = new Dictionary(); private sealed class Key { internal Type Type { get; set; } internal ITypeInspector TypeInspector { get; set; } internal bool EnableStringExampleGeneration { get; set; } internal bool EnableStringFloatCompact { get; set; } internal List CustomFeaturizer { get; set; } public override bool Equals(object obj) { var other = obj as Key; return other != null && this.Type == other.Type && this.TypeInspector == other.TypeInspector && this.EnableStringExampleGeneration == other.EnableStringExampleGeneration && this.EnableStringFloatCompact == other.EnableStringFloatCompact && ((this.CustomFeaturizer == null && other.CustomFeaturizer == null) || this.CustomFeaturizer.SequenceEqual(other.CustomFeaturizer)); } public override int GetHashCode() { return this.Type.GetHashCode() ^ this.TypeInspector.GetHashCode() ^ this.EnableStringExampleGeneration.GetHashCode() ^ this.EnableStringFloatCompact.GetHashCode() ^ (this.CustomFeaturizer == null ? 1 : this.CustomFeaturizer.GetHashCode()); } } /// /// Creates a serializer for the given type and settings. /// /// The user type to serialize. /// /// public static IVowpalWabbitSerializerCompiler CreateSerializer(VowpalWabbitSettings settings = null) { Schema schema = null; Key cacheKey = null; if (settings != null && settings.Schema != null) { schema = settings.Schema; } else { ITypeInspector typeInspector = settings.TypeInspector; if (typeInspector == null) typeInspector = TypeInspector.Default; // only cache non-string generating serializer cacheKey = new Key { Type = typeof(TExample), TypeInspector = typeInspector, CustomFeaturizer = settings == null ? null : settings.CustomFeaturizer, EnableStringExampleGeneration = settings == null ? false : settings.EnableStringExampleGeneration, EnableStringFloatCompact = settings == null ? false : settings.EnableStringFloatCompact }; object serializer; if (SerializerCache.TryGetValue(cacheKey, out serializer)) { return (IVowpalWabbitSerializerCompiler)serializer; } // TODO: enhance caching based on feature list & featurizer set // if no feature mapping is provided, use [Feature] annotation on provided type. schema = typeInspector.CreateSchema(settings, typeof(TExample)); var multiExampleSerializerCompiler = VowpalWabbitMultiExampleSerializerCompiler.TryCreate(settings, schema); if (multiExampleSerializerCompiler != null) return multiExampleSerializerCompiler; } // need at least a single feature to do something sensible if (schema == null || schema.Features.Count == 0) { return null; } var newSerializer = new VowpalWabbitSingleExampleSerializerCompiler( schema, settings == null ? null : settings.CustomFeaturizer, !settings.EnableStringExampleGeneration); if (cacheKey != null) { SerializerCache[cacheKey] = newSerializer; } return newSerializer; } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/VowpalWabbitSingleExampleSerializer.cs000066400000000000000000000303201323713302100300610ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Diagnostics.Contracts; using System.Globalization; using System.Linq; using VW.Labels; using VW.Serializer.Attributes; namespace VW.Serializer { /// /// A serializer from a user type (TExample) to a native Vowpal Wabbit example type. /// /// The source example type. public sealed class VowpalWabbitSingleExampleSerializer : IVowpalWabbitSerializer, IVowpalWabbitExamplePool { private class CacheEntry { internal VowpalWabbitExample Example; internal DateTime LastRecentUse; #if DEBUG internal bool InUse; #endif } private readonly VowpalWabbitSingleExampleSerializerCompiler compiler; private Dictionary exampleCache; #if DEBUG /// /// Reverse lookup from native example to cache entry to enable proper usage. /// /// /// To avoid any performance impact this is only enabled in debug mode. /// private readonly Dictionary reverseLookup; #endif private readonly VowpalWabbit vw; private readonly Action serializerFunc; internal VowpalWabbitSingleExampleSerializer(VowpalWabbitSingleExampleSerializerCompiler compiler, VowpalWabbit vw) { if (compiler == null) throw new ArgumentNullException("compiler"); Contract.Ensures(vw != null); Contract.EndContractBlock(); this.vw = vw; this.compiler = compiler; var exampleType = typeof(TExample); if (!exampleType.IsVisible) throw new ArgumentException($"Type '{typeof(TExample)}' must be public and all enclosing types must be public."); this.serializerFunc = compiler.Func(vw); var cacheableAttribute = (CacheableAttribute) typeof (TExample).GetCustomAttributes(typeof (CacheableAttribute), true).FirstOrDefault(); if (cacheableAttribute == null) return; if (this.vw.Settings.EnableExampleCaching) { if (cacheableAttribute.EqualityComparer == null) this.exampleCache = new Dictionary(); else { if (!typeof(IEqualityComparer).IsAssignableFrom(cacheableAttribute.EqualityComparer)) { throw new ArgumentException( string.Format( CultureInfo.InvariantCulture, "EqualityComparer ({1}) specified in [Cachable] of {0} must implement IEqualityComparer<{0}>", typeof(TExample), cacheableAttribute.EqualityComparer)); } var comparer = (IEqualityComparer)Activator.CreateInstance(cacheableAttribute.EqualityComparer); this.exampleCache = new Dictionary(comparer); } #if DEBUG this.reverseLookup = new Dictionary(new ReferenceEqualityComparer()); #endif } } /// /// True if this instance caches examples, false otherwise. /// public bool CachesExamples { get { return this.exampleCache != null; } } /// /// True if string examples are generated in parallel to native examples. /// public bool EnableStringExampleGeneration { get { return !this.compiler.DisableStringExampleGeneration; } } /// /// Serializes the given to VW string format. /// /// The example to serialize. /// The label to serialize. /// The optional index of the example, the should be attributed to. /// Dictionary used for dictify operation. /// Dictionary used for dictify operation. /// The resulting VW string. public string SerializeToString(TExample example, ILabel label = null, int? index = null, Dictionary dictionary = null, Dictionary fastDictionary = null) { Contract.Requires(example != null); using (var context = new VowpalWabbitMarshalContext(vw, dictionary, fastDictionary)) { this.serializerFunc(context, example, label); return context.ToString(); } } /// /// Serialize the example. /// /// The example to serialize. /// The label to be serialized. /// The optional index of the example, the should be attributed to. /// The serialized example. /// If is annotated using the Cachable attribute, examples are returned from cache. VowpalWabbitExampleCollection IVowpalWabbitSerializer.Serialize(TExample example, ILabel label, int? index) { // dispatch return new VowpalWabbitSingleLineExampleCollection(vw, Serialize(example, label, index)); } /// /// Serialize the example. /// /// The example to serialize. /// The label to be serialized. /// The optional index of the example, the should be attributed to. /// The serialized example. /// If is annotated using the Cachable attribute, examples are returned from cache. public VowpalWabbitExample Serialize(TExample example, ILabel label = null, int? index = null) { Contract.Requires(example != null); Contract.Requires(index == null); if (this.exampleCache == null || label != null) { using (var context = new VowpalWabbitMarshalContext(vw)) { this.serializerFunc(context, example, label); var vwExample = context.ExampleBuilder.CreateExample(); if (this.EnableStringExampleGeneration) vwExample.VowpalWabbitString = context.ToString(); return vwExample; } } CacheEntry result; if (this.exampleCache.TryGetValue(example, out result)) { result.LastRecentUse = DateTime.UtcNow; #if DEBUG if (result.InUse) { throw new ArgumentException("Cached example already in use."); } #endif } else { VowpalWabbitExample nativeExample = null; try { using (var context = new VowpalWabbitMarshalContext(this)) { this.serializerFunc(context, example, label); nativeExample = context.ExampleBuilder.CreateExample(); } result = new CacheEntry { Example = nativeExample, LastRecentUse = DateTime.UtcNow }; this.exampleCache.Add(example, result); #if DEBUG this.reverseLookup.Add(result.Example, result); #endif } catch(Exception e) { if (nativeExample != null) nativeExample.Dispose(); throw e; } } #if DEBUG result.InUse = true; #endif // TODO: support Label != null here and update cached example using new label return result.Example; } /// /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources. /// public void Dispose() { this.Dispose(true); GC.SuppressFinalize(this); } private void Dispose(bool disposing) { if (disposing) { if (this.exampleCache != null) { foreach (var example in this.exampleCache.Values) { example.Example.InnerExample.Dispose(); } this.exampleCache = null; } } } /// /// The associated native instance. /// public VowpalWabbit Native { get { return this.vw; } } /// /// Gets an already allocated instance from the example pool or creates a new one. /// /// public VowpalWabbitExample GetOrCreateNativeExample() { return new VowpalWabbitExample(owner: this, example: this.vw.GetOrCreateNativeExample()); } /// /// Accepts an example back into this pool. /// /// The example to be returned. public void ReturnExampleToPool(VowpalWabbitExample example) { if (this.exampleCache == null) { throw new ObjectDisposedException("VowpalWabbitSerializer"); } #if DEBUG CacheEntry cacheEntry; if (!this.reverseLookup.TryGetValue(example, out cacheEntry)) { throw new ArgumentException("Example is not found in pool"); } if (!cacheEntry.InUse) { throw new ArgumentException("Unused example returned"); } cacheEntry.InUse = false; #endif // if we reach the cache boundary, dispose the oldest example if (this.exampleCache.Count > this.vw.Settings.MaxExampleCacheSize) { var enumerator = this.exampleCache.GetEnumerator(); // this.settings.MaxExampleCacheSize is >= 1 enumerator.MoveNext(); var min = enumerator.Current; while (enumerator.MoveNext()) { if (min.Value.LastRecentUse > enumerator.Current.Value.LastRecentUse) { min = enumerator.Current; } } #if DEBUG this.reverseLookup.Remove(min.Value.Example); #endif this.exampleCache.Remove(min.Key); min.Value.Example.InnerExample.Dispose(); } } private class ReferenceEqualityComparer : IEqualityComparer { public bool Equals(T x, T y) { return object.ReferenceEquals(x, y); } public int GetHashCode(T obj) { return obj.GetHashCode(); } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/Serializer/VowpalWabbitSingleExampleSerializerCompiler.cs000066400000000000000000000655141323713302100315710ustar00rootroot00000000000000using System; using System.Collections.Generic; using System.Diagnostics; using System.Diagnostics.Contracts; using System.Linq; using System.Linq.Expressions; using System.Reflection; using System.Reflection.Emit; using System.Text; using System.Threading.Tasks; using VW.Labels; using VW.Reflection; using VW.Serializer.Intermediate; namespace VW.Serializer { /// /// Compiles a serializers for the given example user type. /// /// The example user type. /// A serializer for the given user example type. public sealed class VowpalWabbitSingleExampleSerializerCompiler : IVowpalWabbitSerializerCompiler { /// /// Internal structure collecting all itmes required to marshal a single feature. /// [DebuggerDisplay("FeatureExpressionInternal(Source={Source}, MarshalMethod={MarshalMethod})")] internal sealed class FeatureExpressionInternal { /// /// The supplied feature expression. /// internal FeatureExpression Source; /// /// The resolved mrarshalling method. /// internal MarshalMethod MarshalMethod; } /// /// Describes the actual marshalling method and the feature type (e.g. ). /// internal sealed class MarshalMethod { /// /// The actual marshalling method. /// internal MethodInfo Method; /// /// The feature type (e.g. ). /// internal Type MetaFeatureType; /// /// True if the method can marshal a full namespace. /// internal bool IsNamespace; } /// /// All discovered features. /// private FeatureExpressionInternal[] allFeatures; private readonly Schema schema; /// /// Ordered list of featurizer types. Marshalling methods are resolved in order of this list. /// is added last as default. /// private readonly List marshallerTypes; /// /// The main body of the serializer holding preemptive calcutions (e.g. . /// private readonly List body; /// /// The body executed for example. /// private readonly List perExampleBody; /// /// Local variables. /// private readonly List variables; /// /// Local variables holding namespaces. /// private readonly List namespaceVariables; /// /// The parameter of the main lambda to . /// private ParameterExpression vwParameter; /// /// The parameter of the main lambda to . /// private ParameterExpression contextParameter; /// /// The parameter of the per example lambda to . /// private ParameterExpression exampleParameter; /// /// The parameter of the per example lambda to /// private ParameterExpression labelParameter; /// /// The list of featurizers. /// private readonly List marshallers; /// /// The list of meta features such as . /// private readonly List metaFeatures; /// /// If true, VowpalWabbit string generation is disabled. /// private readonly bool disableStringExampleGeneration; internal VowpalWabbitSingleExampleSerializerCompiler(Schema schema, IReadOnlyList featurizerTypes, bool disableStringExampleGeneration) { if (schema == null || schema.Features.Count == 0) throw new ArgumentException("schema"); Contract.EndContractBlock(); this.schema = schema; this.disableStringExampleGeneration = disableStringExampleGeneration; this.allFeatures = schema.Features.Select(f => new FeatureExpressionInternal { Source = f }).ToArray(); // collect the types used for marshalling this.marshallerTypes = featurizerTypes == null ? new List() : new List(featurizerTypes); // extract types from overrides defined on particular features var overrideFeaturizerTypes = schema.Features .Select(f => f.OverrideSerializeMethod) .Where(o => o != null) .Select(o => o.DeclaringType); this.marshallerTypes.AddRange(overrideFeaturizerTypes); // add as last this.marshallerTypes.Add(typeof(VowpalWabbitDefaultMarshaller)); this.body = new List(); this.perExampleBody = new List(); this.variables = new List(); this.namespaceVariables = new List(); this.marshallers = new List(); this.metaFeatures = new List(); this.CreateMarshallers(); this.ResolveFeatureMarshallingMethods(); this.CreateParameters(); this.CreateLabel(); this.CreateNamespacesAndFeatures(); this.CreateLambdas(); this.Func = (Func>)this.SourceExpression.CompileToFunc(); } internal bool DisableStringExampleGeneration { get { return this.disableStringExampleGeneration; } } /// /// Creates a bound serializers. /// /// The vw instance to bind to. /// IVowpalWabbitSerializer IVowpalWabbitSerializerCompiler.Create(VowpalWabbit vw) { return this.Create(vw); } /// /// Creates a serializer for bound to . /// /// The VW native instance examples will be assocated with. /// A serializer for . public VowpalWabbitSingleExampleSerializer Create(VowpalWabbit vw) { return new VowpalWabbitSingleExampleSerializer(this, vw); } private void CreateLabel() { // CODE if (labelParameter == null) this.perExampleBody.Add(Expression.IfThen( Expression.NotEqual(this.labelParameter, Expression.Constant(null, typeof(ILabel))), this.CreateMarshallerCall("MarshalLabel", this.contextParameter, this.labelParameter))); var label = this.schema.Label; if (label != null) { // CODE condition1 && condition2 && condition3 ... var condition = label.ValueValidExpressionFactories .Skip(1) .Aggregate( label.ValueValidExpressionFactories.First()(this.exampleParameter), (cond, factory) => Expression.AndAlso(cond, factory(this.exampleParameter))); // CODE if (labelParameter != null && example.Label != null && ...) this.perExampleBody.Add( Expression.IfThen( Expression.AndAlso( Expression.Equal(this.labelParameter, Expression.Constant(null, typeof(ILabel))), condition), // CODE MarshalLabel(context, example.Label) this.CreateMarshallerCall("MarshalLabel", this.contextParameter, label.ValueExpressionFactory(this.exampleParameter)))); } } /// /// Define variables and instantiate marshaller types. /// private void CreateMarshallers() { foreach (var marshallerType in this.marshallerTypes) { var marshaller = Expression.Parameter(marshallerType, "marshaller_" + marshallerType.Name); this.marshallers.Add(marshaller); this.variables.Add(marshaller); // CODE new FeaturizerType(disableStringExampleGeneration) var newExpr = CreateNew(marshallerType, Expression.Constant(disableStringExampleGeneration)); if (newExpr == null) { // CODE new MarshallerType() newExpr = Expression.New(marshallerType); } // marshaller = new ... this.body.Add(Expression.Assign(marshaller, newExpr)); } } private MarshalMethod ResolveFeatureMarshalMethod(FeatureExpression feature) { if (feature.OverrideSerializeMethod != null) { return new MarshalMethod { Method = feature.OverrideSerializeMethod, MetaFeatureType = feature.OverrideSerializeMethod.GetParameters().Select(p => p.ParameterType).First(t => typeof(Feature).IsAssignableFrom(t)) }; } string methodName; Type[] metaFeatureTypeCandidates; if (feature.FeatureType == typeof(string)) { switch(feature.StringProcessing) { case StringProcessing.Escape: methodName = "MarshalFeatureStringEscape"; break; case StringProcessing.EscapeAndIncludeName: methodName = "MarshalFeatureStringEscapeAndIncludeName"; break; case StringProcessing.Split: methodName = "MarshalFeatureStringSplit"; break; default: throw new ArgumentException("feature.StringProcessing is not supported: " + feature.StringProcessing); } metaFeatureTypeCandidates = new [] { typeof(Feature) }; } else if (feature.FeatureType.IsEnum) { methodName = "MarshalEnumFeature"; metaFeatureTypeCandidates = new [] { typeof(EnumerizedFeature<>).MakeGenericType(feature.FeatureType) }; } else if (feature.Enumerize) { methodName = "MarshalEnumerizeFeature"; metaFeatureTypeCandidates = new [] { typeof(Feature) }; } else { // probe for PreHashedFeature marshal method, than fallback methodName = "MarshalFeature"; metaFeatureTypeCandidates = new [] { typeof(PreHashedFeature), typeof(Feature) }; } // remove Nullable<> from feature type var featureType = feature.FeatureType; if(featureType.IsGenericType && featureType.GetGenericTypeDefinition() == typeof(Nullable<>)) { featureType = featureType.GetGenericArguments()[0]; } var method = ResolveFeatureMarshalMethod("MarshalNamespace", metaFeatureTypeCandidates, featureType, isNamespace: true); if (method == null) method = ResolveFeatureMarshalMethod(methodName, metaFeatureTypeCandidates, featureType, isNamespace: false); return method; } private MarshalMethod ResolveFeatureMarshalMethod(string methodName, Type[] metaFeatureTypeCandidates, Type featureType, bool isNamespace) { foreach (var metaFeatureType in metaFeatureTypeCandidates) { // find visitor.(VowpalWabbitMarshallingContext context, Namespace ns, feature, value) var method = this.marshallerTypes .Select(visitor => ReflectionHelper.FindMethod( visitor, methodName, typeof(VowpalWabbitMarshalContext), typeof(Namespace), metaFeatureType, featureType)) .FirstOrDefault(m => m != null); if (method != null) return new MarshalMethod { Method = method, MetaFeatureType = metaFeatureType, IsNamespace = isNamespace }; } return null; } private bool ContainsAncestor(FeatureExpressionInternal candidate, List validFeature) { if (candidate.Source.Parent == null) return false; if (validFeature.Any(valid => object.ReferenceEquals(valid.Source, candidate.Source.Parent))) return true; var parent = this.allFeatures.First(f => object.ReferenceEquals(f.Source, candidate.Source.Parent)); return ContainsAncestor(parent, validFeature); } /// /// Resolve methods for each feature base on feature type and configuration. /// private void ResolveFeatureMarshallingMethods() { var validFeature = new List(this.allFeatures.Length); foreach (var feature in this.allFeatures) { // skip any feature which parent feature is already resolved if (ContainsAncestor(feature, validFeature)) continue; feature.MarshalMethod = this.ResolveFeatureMarshalMethod(feature.Source); if (feature.MarshalMethod != null) validFeature.Add(feature); } this.allFeatures = validFeature.ToArray(); } /// /// define functions input parameter /// private void CreateParameters() { this.vwParameter = Expression.Parameter(typeof(VowpalWabbit), "vw"); this.contextParameter = Expression.Parameter(typeof(VowpalWabbitMarshalContext), "context"); this.exampleParameter = Expression.Parameter(typeof(TExample), "example"); this.labelParameter = Expression.Parameter(typeof(ILabel), "label"); } /// /// Instantiate the meta information object such as /// for a given feature. /// /// The feature. /// The namespace. /// The "new" expression for the meta information object. private Expression CreateFeature(FeatureExpressionInternal featureInternal, Expression @namespace) { FeatureExpression feature = featureInternal.Source; var metaFeatureType = featureInternal.MarshalMethod.MetaFeatureType; if (metaFeatureType.IsGenericType && metaFeatureType.GetGenericTypeDefinition() == typeof(EnumerizedFeature<>)) { // preemptively calculate all hashes for each enum value var featureParameter = Expression.Parameter(metaFeatureType); var valueParameter = Expression.Parameter(feature.FeatureType); var body = new List(); var hashVariables = new List(); foreach (var value in Enum.GetValues(feature.FeatureType)) { var hashVar = Expression.Variable(typeof(UInt64)); hashVariables.Add(hashVar); // CODE hashVar = feature.FeatureHashInternal(value); body.Add(Expression.Assign(hashVar, Expression.Call(featureParameter, metaFeatureType.GetMethod("FeatureHashInternal"), Expression.Constant(value)))); } var cases = Enum.GetValues(feature.FeatureType) .Cast() .Zip(hashVariables, (value, hash) => Expression.SwitchCase( hash, Expression.Constant(value, feature.FeatureType))) .ToArray(); // expand the switch(value) { case enum1: return hash1; .... } var hashSwitch = Expression.Switch(valueParameter, Expression.Block(Expression.Throw(Expression.New(typeof(NotSupportedException))), Expression.Constant((UInt64)0, typeof(UInt64))), cases); // CODE return value => switch(value) { .... } body.Add(Expression.Lambda(hashSwitch, valueParameter)); return CreateNew( metaFeatureType, this.vwParameter, @namespace, Expression.Constant(feature.Name, typeof(string)), Expression.Constant(feature.AddAnchor), Expression.Constant(feature.Dictify), Expression.Lambda(Expression.Block(hashVariables, body), featureParameter)); } else if (metaFeatureType == typeof(PreHashedFeature)) { // CODE new PreHashedFeature(vw, namespace, "Name", "AddAnchor"); return CreateNew( typeof(PreHashedFeature), this.vwParameter, @namespace, Expression.Constant(feature.Name, typeof(string)), Expression.Constant(feature.AddAnchor), Expression.Constant(feature.Dictify)); } else // CODE new Feature("Name", ...) return CreateNew( metaFeatureType, Expression.Constant(feature.Name, typeof(string)), Expression.Constant(feature.AddAnchor), Expression.Constant(feature.Dictify)); } /// /// Helper to create the "new" expression using a matching constructor. /// /// The type of the new object. /// The actual parameters for the constructor. /// The "new" expression bound to . private static Expression CreateNew(Type type, params Expression[] constructorParameters) { var ctor = type.GetConstructor(constructorParameters.Select(e => e.Type).ToArray()); if (ctor == null) return null; return Expression.New(ctor, constructorParameters); } private void CreateNamespacesAndFeatures() { var featuresByNamespace = this.allFeatures .GroupBy( f => new { f.Source.Namespace, f.Source.FeatureGroup }, f => f); foreach (var ns in featuresByNamespace) { // each feature can have 2 additional parameters (namespace + feature) // Visit(VowpalWabbit, CustomNamespace, CustomFeature) // create default namespace object var namespaceVariable = Expression.Variable(typeof(Namespace), "ns_" + ns.Key.FeatureGroup + ns.Key.Namespace); this.variables.Add(namespaceVariable); // CODE ns = new Namespace(vw, name, featureGroup); this.body.Add(Expression.Assign(namespaceVariable, CreateNew( typeof(Namespace), this.vwParameter, Expression.Constant(ns.Key.Namespace, typeof(string)), ns.Key.FeatureGroup == null ? (Expression)Expression.Constant(null, typeof(char?)) : Expression.New((ConstructorInfo)ReflectionHelper.GetInfo((char v) => new char?(v)), Expression.Constant((char)ns.Key.FeatureGroup))))); var fullNamespaceCalls = new List(); var featureVisits = new List(ns.Count()); foreach (var feature in ns.OrderBy(f => f.Source.Order)) { var newFeature = feature.Source.FeatureExpressionFactory != null ? feature.Source.FeatureExpressionFactory(this.vwParameter, namespaceVariable) : this.CreateFeature(feature, namespaceVariable); var featureVariable = Expression.Variable(newFeature.Type, "feature_" + feature.Source.Name); this.variables.Add(featureVariable); // CODE var feature = new ... this.body.Add(Expression.Assign(featureVariable, newFeature)); // TODO: optimize var marshaller = this.marshallers.First(f => f.Type == feature.MarshalMethod.Method.ReflectedType); var valueVariable = feature.Source.ValueExpressionFactory(this.exampleParameter); Expression featureVisit; if (feature.Source.IsNullable) { // if (value != null) featurizer.MarshalXXX(vw, context, ns, feature, (FeatureType)value); featureVisit = Expression.IfThen( Expression.NotEqual(valueVariable, Expression.Constant(null)), Expression.Call( marshaller, feature.MarshalMethod.Method, this.contextParameter, namespaceVariable, featureVariable, Expression.Convert(valueVariable, feature.Source.FeatureType))); } else { // featurizer.MarshalXXX(vw, context, ns, feature, value); featureVisit = Expression.Call( marshaller, feature.MarshalMethod.Method, this.contextParameter, namespaceVariable, featureVariable, valueVariable); } if (feature.Source.ValueValidExpressionFactories != null && feature.Source.ValueValidExpressionFactories.Count > 0) { // CODE condition1 && condition2 && condition3 ... var condition = feature.Source.ValueValidExpressionFactories .Skip(1) .Aggregate( feature.Source.ValueValidExpressionFactories.First()(this.exampleParameter), (cond, factory) => Expression.AndAlso(cond, factory(this.exampleParameter))); featureVisit = Expression.IfThen(condition, featureVisit); } if (feature.MarshalMethod.IsNamespace) this.perExampleBody.Add(featureVisit); else featureVisits.Add(featureVisit); } if (featureVisits.Count > 0) { var featureVisitLambda = Expression.Lambda(Expression.Block(featureVisits)); // CODE: featurizer.MarshalNamespace(context, namespace, { ... }) this.perExampleBody.Add(this.CreateMarshallerCall("MarshalNamespace", this.contextParameter, namespaceVariable, featureVisitLambda)); } } } /// /// Create the invocation expression of a marshalling method. /// /// The marshalling method to invoke. /// The parameters for this method. private MethodCallExpression CreateMarshallerCall(string methodName, params Expression[] parameters) { var parameterTypes = parameters.Select(p => p.Type).ToArray(); foreach (var marshaller in this.marshallers) { var method = marshaller.Type.GetMethod(methodName, parameterTypes); if (method != null) { return Expression.Call(marshaller, method, parameters); } } throw new ArgumentException("Unable to find MarshalNamespace(VowpalWabbitMarshallingContext, Namespace, Action) on any featurizer"); } /// /// Creates the main lambda and the per example lambda. /// private void CreateLambdas() { // CODE (TExample, Label) => { ... } this.body.Add( Expression.Lambda( typeof(Action), Expression.Block(this.perExampleBody), this.contextParameter, this.exampleParameter, this.labelParameter)); // CODE return (vw) => { ... return (ex, label) => { ... } } this.SourceExpression = Expression.Lambda>>( Expression.Block(this.variables, this.body), this.vwParameter); } /// /// The source expression tree is built from. /// public Expression>> SourceExpression { get; private set; } /// /// The closure used for serialization. /// public Func> Func { get; private set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/VowpalWabbit.cs000066400000000000000000000524341323713302100212520ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Diagnostics.Contracts; using VW.Labels; using VW.Serializer; namespace VW { /// /// VW wrapper supporting data ingest using declarative serializer infrastructure. /// /// The user type to be serialized. public class VowpalWabbit : IDisposable { /// /// Native vw instance. /// private VowpalWabbit vw; /// /// The example serializer. /// private IVowpalWabbitSerializer serializer; /// /// The example serializer compilation. Useful when debugging. /// private IVowpalWabbitSerializerCompiler compiledSerializer; /// /// The serializer used for learning. It's only set if the serializer is non-caching. /// By having a second field there is one less check that has to be done in the hot path. /// private readonly IVowpalWabbitSerializer learnSerializer; private readonly VowpalWabbitSingleExampleSerializer singleLineSerializer; /// /// Initializes a new instance of the class. /// /// Command line arguments passed to native instance. public VowpalWabbit(String args) : this(new VowpalWabbit(args)) { } /// /// Initializes a new instance of the class. /// /// Arguments passed to native instance. public VowpalWabbit(VowpalWabbitSettings settings) : this(new VowpalWabbit(settings)) { } /// /// Initializes a new instance of the class. /// /// The native instance to wrap. /// This instance takes ownership of instance and disposes it. public VowpalWabbit(VowpalWabbit vw) : this(vw, VowpalWabbitSerializerFactory.CreateSerializer(vw.Settings)) { } /// /// Initializes a new instance of the class. /// /// The native instance to wrap. /// The per-compiled serializer. /// This instance takes ownership of instance and disposes it. public VowpalWabbit(VowpalWabbit vw, IVowpalWabbitSerializerCompiler compiledSerializer) { if (vw == null) throw new ArgumentNullException(nameof(vw)); if (compiledSerializer == null) throw new ArgumentNullException(nameof(compiledSerializer)); Contract.Ensures(this.serializer != null); Contract.EndContractBlock(); this.vw = vw; this.compiledSerializer = compiledSerializer; this.serializer = this.compiledSerializer.Create(vw); // have a 2nd member to throw NullReferenceException in release instead of silently producing wrong results. this.learnSerializer = this.serializer.CachesExamples ? null : this.serializer; // have a 3rd member to avoid cast everytime... this.singleLineSerializer = this.serializer as VowpalWabbitSingleExampleSerializer; } /// /// The wrapped VW instance. /// public VowpalWabbit Native { get { return this.vw; } } /// /// The serializer used to marshal examples. /// public IVowpalWabbitSerializerCompiler Serializer { get { return this.compiledSerializer; } } /// /// Learns from the given example. /// /// The example to learn. /// The label for this . /// The optional index of the example, the should be attributed to. public void Learn(TExample example, ILabel label = null, int? index = null) { Contract.Requires(example != null); Contract.Requires(label != null); #if DEBUG if (this.serializer.CachesExamples) { throw new NotSupportedException("Cached examples cannot be used for learning"); } #endif // in release this throws NullReferenceException instead of producing silently wrong results using (var ex = this.learnSerializer.Serialize(example, label, index)) { ex.Learn(); } } /// /// Learn from the given example and returns the current prediction for it. /// /// The prediction type. /// The example to learn. /// The label for this . /// The prediction factory to be used. See . /// The prediction for the given . public TPrediction Learn(TExample example, ILabel label, IVowpalWabbitPredictionFactory predictionFactory) { Contract.Requires(example != null); Contract.Requires(label != null); Contract.Requires(predictionFactory != null); #if DEBUG // only in debug, since it's a hot path if (this.serializer.CachesExamples) { throw new NotSupportedException("Cached examples cannot be used for learning"); } #endif using (var ex = this.learnSerializer.Serialize(example, label)) { return ex.Learn(predictionFactory); } } /// /// Predicts for the given example. /// /// The example to predict for. /// This label can be used to weight the example. public void Predict(TExample example, ILabel label = null) { Contract.Requires(example != null); using (var ex = this.serializer.Serialize(example, label)) { ex.Predict(); } } /// /// Predicts for the given example. /// /// The prediction type. /// The example to predict for. /// The prediction factory to be used. See . /// This label can be used to weight the example. public TPrediction Predict(TExample example, IVowpalWabbitPredictionFactory predictionFactory, ILabel label = null) { Contract.Requires(example != null); Contract.Requires(predictionFactory != null); using (var ex = this.serializer.Serialize(example, label)) { return ex.Predict(predictionFactory); } } /// /// Learn from the given example and return the current prediction for it. /// /// The action dependent features. /// The index of the example to learn within . /// The label for the example to learn. public void Learn(IReadOnlyCollection actionDependentFeatures, int index, ILabel label) { Contract.Requires(actionDependentFeatures != null); Contract.Requires(this.singleLineSerializer != null, string.Format( "{0} maps to a multiline example. Use VowpalWabbit.Learn<{0}>({0} example,...) instead.", typeof(TExample))); VowpalWabbitMultiLine.Learn( this.vw, null, this.singleLineSerializer, null, actionDependentFeatures, index, label); } /// /// Learn from the given example and return the current prediction for it. /// /// The action dependent features. /// The index of the example to learn within . /// The label for the example to learn. /// The ranked prediction for the given examples. public ActionDependentFeature[] LearnAndPredict(IReadOnlyCollection actionDependentFeatures, int index, ILabel label) { Contract.Requires(actionDependentFeatures != null); Contract.Requires(index >= 0); Contract.Requires(label != null); Contract.Requires(this.singleLineSerializer != null, string.Format( "{0} maps to a multiline example. Use VowpalWabbit.Learn<{0}>({0} example,...) instead.", typeof(TExample))); return VowpalWabbitMultiLine.LearnAndPredict( this.vw, null, this.singleLineSerializer, null, actionDependentFeatures, index, label); } /// /// Predict for the given example and return the current prediction for it. /// /// The action dependent features. /// The index of the example to evaluate within . /// The label for the example to evaluate. /// The ranked prediction for the given examples. public ActionDependentFeature[] Predict(IReadOnlyCollection actionDependentFeatures, int? index = null, ILabel label = null) { Contract.Requires(actionDependentFeatures != null); Contract.Requires(this.singleLineSerializer != null, string.Format( "{0} maps to a multiline example. Use VowpalWabbit.Learn<{0}>({0} example,...) instead.", typeof(TExample))); return VowpalWabbitMultiLine.Predict( this.vw, null, this.singleLineSerializer, null, actionDependentFeatures, index, label); } /// /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources. /// public void Dispose() { this.Dispose(true); GC.SuppressFinalize(this); } private void Dispose(bool disposing) { if (disposing) { if (this.serializer != null) { this.serializer.Dispose(); this.serializer = null; } if (this.vw != null) { this.vw.Dispose(); this.vw = null; } } } } /// /// VW wrapper for multiline ingest. /// /// The user type of the shared feature. /// The user type for each action dependent feature. public class VowpalWabbit : IDisposable { /// /// The native wrapper. /// private VowpalWabbit vw; /// /// The shared example serializer. /// private VowpalWabbitSingleExampleSerializer serializer; /// /// The action dependent feature serializer. /// private VowpalWabbitSingleExampleSerializer actionDependentFeatureSerializer; /// /// The action dependent feature serializer valid for learning. If example caching is enabled, this is null. /// private readonly VowpalWabbitSingleExampleSerializer actionDependentFeatureLearnSerializer; /// /// Initializes a new instance of the class. /// /// Command line arguments passed to native instance. public VowpalWabbit(String args) : this(new VowpalWabbit(args)) { } /// /// Initializes a new instance of the class. /// /// Arguments passed to native instance. public VowpalWabbit(VowpalWabbitSettings settings) : this(new VowpalWabbit(settings)) { } /// /// Initializes a new instance of the class. /// /// The native instance to wrap. /// This instance takes ownership of instance and disposes it. public VowpalWabbit(VowpalWabbit vw) { if (vw == null) { throw new ArgumentNullException("vw"); } Contract.EndContractBlock(); this.vw = vw; this.serializer = VowpalWabbitSerializerFactory.CreateSerializer(vw.Settings).Create(vw) as VowpalWabbitSingleExampleSerializer; if (this.serializer == null) throw new ArgumentException(string.Format( "{0} maps to a multiline example. Use VowpalWabbit<{0}> instead.", typeof(TExample))); var adfSettings = (VowpalWabbitSettings)vw.Settings.Clone(); adfSettings.Schema = vw.Settings.ActionDependentSchema; this.actionDependentFeatureSerializer = VowpalWabbitSerializerFactory.CreateSerializer(adfSettings).Create(vw) as VowpalWabbitSingleExampleSerializer; if (this.actionDependentFeatureSerializer == null) throw new ArgumentException(string.Format( "{0} maps to a multiline example. Use VowpalWabbit<{0}> instead.", typeof(TActionDependentFeature))); // have a 2nd member to throw NullReferenceException in release instead of silently producing wrong results. this.actionDependentFeatureLearnSerializer = this.actionDependentFeatureSerializer.CachesExamples ? null : this.actionDependentFeatureSerializer; } /// /// The wrapped VW instance. /// public VowpalWabbit Native { get { return this.vw; } } /// /// Internal example serializer. /// internal IVowpalWabbitSerializer ExampleSerializer { get { return this.serializer; } } /// /// Internal action dependent feature serializer. /// internal IVowpalWabbitSerializer ActionDependentFeatureSerializer { get { return this.actionDependentFeatureSerializer; } } /// /// Learn from the given example and return the current prediction for it. /// /// The shared example. /// The action dependent features. /// The index of the example to learn within . /// The label for the example to learn. public void Learn(TExample example, IReadOnlyCollection actionDependentFeatures, int index, ILabel label) { Contract.Requires(example != null); Contract.Requires(actionDependentFeatures != null); VowpalWabbitMultiLine.Learn( this.vw, this.serializer, this.actionDependentFeatureLearnSerializer, example, actionDependentFeatures, index, label); } /// /// Learn from the given example and return the current prediction for it. /// /// The shared example. /// The action dependent features. /// The index of the example to learn within . /// The label for the example to learn. /// The ranked prediction for the given examples. public ActionDependentFeature[] LearnAndPredict(TExample example, IReadOnlyCollection actionDependentFeatures, int index, ILabel label) { Contract.Requires(example != null); Contract.Requires(actionDependentFeatures != null); Contract.Requires(index >= 0); Contract.Requires(label != null); return VowpalWabbitMultiLine.LearnAndPredict( this.vw, this.serializer, this.actionDependentFeatureLearnSerializer, example, actionDependentFeatures, index, label); } /// /// Learn from the given example and return the current prediction for it. /// /// The shared example. /// The action dependent features. /// The index of the example to evaluate within . /// The label for the example to evaluate. /// The ranked prediction for the given examples. public ActionDependentFeature[] Predict(TExample example, IReadOnlyCollection actionDependentFeatures, int? index = null, ILabel label = null) { Contract.Requires(example != null); Contract.Requires(actionDependentFeatures != null); return VowpalWabbitMultiLine.Predict( this.vw, this.serializer, this.actionDependentFeatureSerializer, example, actionDependentFeatures, index, label); } /// /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources. /// public void Dispose() { this.Dispose(true); GC.SuppressFinalize(this); } private void Dispose(bool disposing) { if (disposing) { if (this.vw != null) { this.vw.Dispose(); this.vw = null; } if (this.serializer != null) { this.serializer.Dispose(); this.serializer = null; } if (this.actionDependentFeatureSerializer != null) { this.actionDependentFeatureSerializer.Dispose(); this.actionDependentFeatureSerializer = null; } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/VowpalWabbitDynamic.cs000066400000000000000000000157571323713302100225660ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Diagnostics.Contracts; using System.Linq; using System.Linq.Expressions; using System.Reflection; using System.Text; using System.Threading.Tasks; using VW.Labels; using VW.Serializer; namespace VW { /// /// Vowpal Wabbit wrapper for anonymous classes. Type used for serialization doesn't need to be known at compile time, /// but it's checked at runtime. /// /// For each call to there is additional overhead as the type is looked up in a dictionary compared to . public class VowpalWabbitDynamic : IDisposable { private Dictionary serializers; private Dictionary serializeMethods; private VowpalWabbit vw; /// /// Initializes a new instance of the class. /// /// Command line arguments passed to native instance. public VowpalWabbitDynamic(string arguments) : this(new VowpalWabbitSettings(arguments)) { } /// /// Initializes a new instance of the class. /// /// Arguments passed to native instance. public VowpalWabbitDynamic(VowpalWabbitSettings settings) { this.vw = new VowpalWabbit(settings); this.serializers = new Dictionary(); this.serializeMethods = new Dictionary(); } private VowpalWabbitExampleCollection SerializeTyped(T example, ILabel label, int? index) { IDisposable serializer; if (!this.serializers.TryGetValue(typeof(T), out serializer)) { var serializerCompiler = VowpalWabbitSerializerFactory.CreateSerializer(this.vw.Settings); if (serializerCompiler == null) throw new ArgumentException("No feature discovered for type: " + typeof(T)); serializer = serializerCompiler.Create(this.vw); this.serializers.Add(typeof(T), serializer); } return ((IVowpalWabbitSerializer)serializer).Serialize(example, label, index); } private VowpalWabbitExampleCollection Serialize(object example, ILabel label = null, int? index = null) { var type = example.GetType(); MethodInfo method; if (!this.serializeMethods.TryGetValue(type, out method)) { method = typeof(VowpalWabbitDynamic) .GetMethod("SerializeTyped", BindingFlags.Instance | BindingFlags.NonPublic) .MakeGenericMethod(type); this.serializeMethods.Add(type, method); } return (VowpalWabbitExampleCollection)method.Invoke(this, new[] { example, label, index }); } /// /// Learns from the given example. /// /// The example to learn. /// The optional label for this . /// The optional index of the example, the should be attributed to. public void Learn(object example, ILabel label = null, int? index = null) { Contract.Requires(example != null); using (var ex = this.Serialize(example, label, index)) { ex.Learn(); } } /// /// Learns from the given example and returns the current prediction. /// /// The example to learn. /// The prediction factory used to extract the prediction. Use . /// The optional label for this . /// The optional index of the example, the should be attributed to. public TPrediction Learn(object example, IVowpalWabbitPredictionFactory predictionFactory, ILabel label = null, int? index = null) { Contract.Requires(example != null); Contract.Requires(predictionFactory != null); using (var ex = this.Serialize(example, label, index)) { return ex.Learn(predictionFactory); } } /// /// Predict for the given example and return the current prediction for it. /// /// /// /// /// The optional index of the example to evaluate within /// The optional label for the example to evaluate. /// public TPrediction Predict(object example, IVowpalWabbitPredictionFactory predictionFactory, ILabel label = null, int? index = null) { Contract.Requires(example != null); Contract.Requires(predictionFactory != null); using (var ex = this.Serialize(example, label, index)) { return ex.Predict(predictionFactory); } } /// /// The wrapped VW instance. /// public VowpalWabbit Native { get { return this.vw; } } /// /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources. /// public void Dispose() { this.Dispose(true); GC.SuppressFinalize(this); } private void Dispose(bool disposing) { if (disposing) { if (this.serializers != null) { foreach (var serializer in this.serializers) serializer.Value.Dispose(); this.serializers = null; } if (this.vw != null) { this.vw.Dispose(); this.vw = null; } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/VowpalWabbitExampleCollection.cs000066400000000000000000000137721323713302100246040ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Diagnostics.Contracts; using System.Linq; using System.Text; using System.Threading.Tasks; using VW.Labels; namespace VW { /// /// Base class for JSON deserialization result. /// public abstract class VowpalWabbitExampleCollection : IDisposable { /// /// The native VW instance. /// private readonly VowpalWabbit vw; /// /// Initializes a new instance of the class. /// /// The VW native instance. protected VowpalWabbitExampleCollection(VowpalWabbit vw) { Contract.Requires(vw != null); this.vw = vw; } /// /// Learns this example on the VW instance used for marshalling or the optionally passed on . /// /// The optional VW instance used for learning. Defaults to the one used for marshalling. public void Learn(VowpalWabbit vw = null) { this.LearnInternal(vw ?? this.vw); } /// /// Predicts for this example. /// /// Use this VW instance for prediction instead of the one the example was created from. public void Predict(VowpalWabbit vw = null) { this.PredictInternal(vw ?? this.vw); } /// /// Learn from this example and returns the current prediction for it. /// /// The prediction type. /// The prediction factory to be used. See . /// The prediction for the this example. /// Use this VW instance for learning instead of the one the example was created from. public TPrediction Learn(IVowpalWabbitPredictionFactory predictionFactory, VowpalWabbit vw = null) { return this.LearnInternal(predictionFactory, vw ?? this.vw); } /// /// Predicts for this example and returns the current prediction for it. /// /// The prediction type. /// The prediction factory to be used. See . /// The prediction for the this example. /// Use this VW instance for prediction instead of the one the example was created from. public TPrediction Predict(IVowpalWabbitPredictionFactory predictionFactory, VowpalWabbit vw = null) { return this.PredictInternal(predictionFactory, vw ?? this.vw); } /// /// Learns from this example. /// /// Use this VW instance for learning instead of the one the example was created from. protected abstract void LearnInternal(VowpalWabbit vw); /// /// Predicts for this example. /// /// Use this VW instance for prediction instead of the one the example was created from. protected abstract void PredictInternal(VowpalWabbit vw); /// /// Learn from this example and returns the current prediction for it. /// /// The prediction type. /// The prediction factory to be used. See . /// The prediction for the this example. /// Use this VW instance for learning instead of the one the example was created from. protected abstract TPrediction LearnInternal(IVowpalWabbitPredictionFactory predictionFactory, VowpalWabbit vw); /// /// Predicts for this example and returns the current prediction for it. /// /// The prediction type. /// The prediction factory to be used. See . /// The prediction for the this example. /// Use this VW instance for prediction instead of the one the example was created from. protected abstract TPrediction PredictInternal(IVowpalWabbitPredictionFactory predictionFactory, VowpalWabbit vw); /// /// The optional string version of the example. /// public abstract string VowpalWabbitString { get; } /// /// The number of feature this example holds. /// public abstract ulong NumberOfFeatures { get; } /// /// All labels this example holds. /// public abstract IEnumerable Labels { get; } /// /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources. /// public abstract void Dispose(); } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/VowpalWabbitMultiLine.cs000066400000000000000000000533301323713302100230710ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Diagnostics.Contracts; using System.Linq; using System.Text; using VW.Labels; using VW.Serializer; namespace VW { /// /// Helper class to properly feed multi-line examples into vw. /// public static class VowpalWabbitMultiLine { /// /// Serializes the specifed example to VW native string format. /// /// The user example type. /// The user action dependent feature type. /// The VW instance. /// The shared example. /// The action dependent features. /// The optional index of the label example. /// The optional label. /// Used to extract features into dictionary. /// Used to extract features into dictionary. This should use a faster comparison mehtod (e.g. reference equals). /// The string serialized example. public static string SerializeToString( VowpalWabbit vw, TExample example, IReadOnlyCollection actionDependentFeatures, int? index = null, ILabel label = null, Dictionary dictionary = null, Dictionary fastDictionary = null) { #if DEBUG if (!vw.Native.Settings.EnableStringExampleGeneration) { throw new ArgumentException("vw.Settings.EnableStringExampleGeneration must be enabled"); } #endif return SerializeToString( vw.Native, example, actionDependentFeatures, index, label, vw.ExampleSerializer, vw.ActionDependentFeatureSerializer, dictionary, fastDictionary); } /// /// Serializes the specifed example to VW native string format. /// /// The user example type. /// The user action dependent feature type. /// The VW instance. /// The shared example. /// The action dependent features. /// The optional index of the label example. /// The optional label. /// The example serializer. /// The action dependent feature serializer. /// Dictionary used for dictify operation. /// Dictionary used for dictify operation. /// The string serialized example. public static string SerializeToString( VowpalWabbit vw, TExample example, IReadOnlyCollection actionDependentFeatures, int? index = null, ILabel label = null, IVowpalWabbitSerializer serializer = null, IVowpalWabbitSerializer actionDependentFeatureSerializer = null, Dictionary dictionary = null, Dictionary fastDictionary = null) { if (vw == null) throw new ArgumentNullException("vw"); if (serializer == null) { serializer = VowpalWabbitSerializerFactory.CreateSerializer(new VowpalWabbitSettings { EnableStringExampleGeneration = true }).Create(vw); } else if (!serializer.EnableStringExampleGeneration) { throw new ArgumentException("Serializer must be compiled using EnableStringExampleGeneration = true"); } if (actionDependentFeatureSerializer == null) { actionDependentFeatureSerializer = VowpalWabbitSerializerFactory.CreateSerializer(new VowpalWabbitSettings { EnableStringExampleGeneration = true }).Create(vw); } else if (!actionDependentFeatureSerializer.EnableStringExampleGeneration) { throw new ArgumentException("Action dependent serializer must be compiled using EnableStringExampleGeneration = true"); } var stringExample = new StringBuilder(); var sharedExample = serializer.SerializeToString(example, SharedLabel.Instance, null, dictionary, fastDictionary); // check if we have shared features if (!string.IsNullOrWhiteSpace(sharedExample)) { stringExample.AppendLine(sharedExample); } var i = 0; foreach (var actionDependentFeature in actionDependentFeatures) { var adfExample = actionDependentFeatureSerializer.SerializeToString(actionDependentFeature, index != null && i == index ? label : null, null, dictionary, fastDictionary); if (!string.IsNullOrWhiteSpace(adfExample)) { stringExample.AppendLine(adfExample); } i++; } return stringExample.ToString(); } /// /// A named delegate for the action to be taken once all the examples are marshalled. /// /// The action dependent feature user type. /// Marshalled valid examples. /// List of valid marshalled examples. /// List of empty non-marshalled examples. public delegate void LearnOrPredictAction( IReadOnlyList validExamples, IReadOnlyList> validActionDependentFeatures, IReadOnlyList> emptyActionDependentFeatures); /// /// Simplify learning of examples with action dependent features. /// /// User example type. /// Action dependent feature type. /// The VowpalWabbit instances. /// The example serializer. /// The action dependent feature serializer. /// The example. /// The action dependent features. /// An action executed once the set of valid examples is determined. /// The optional index of the action dependent feature this label belongs too. /// The optional label to be used for learning or evaluation. public static void Execute( VowpalWabbit vw, VowpalWabbitSingleExampleSerializer serializer, VowpalWabbitSingleExampleSerializer actionDependentFeatureSerializer, TExample example, IReadOnlyCollection actionDependentFeatures, LearnOrPredictAction predictOrLearn, int? index = null, ILabel label = null) { Contract.Requires(vw != null); Contract.Requires(actionDependentFeatureSerializer != null); Contract.Requires(example != null); Contract.Requires(actionDependentFeatures != null); var examples = new List(actionDependentFeatures.Count + 1); var validExamples = new List(actionDependentFeatures.Count + 1); var validActionDependentFeatures = new List>(actionDependentFeatures.Count + 1); var emptyActionDependentFeatures = new List>(actionDependentFeatures.Count + 1); VowpalWabbitExample emptyExample = null; try { // contains prediction results if (serializer != null) { var sharedExample = serializer.Serialize(example, SharedLabel.Instance); // check if we have shared features if (sharedExample != null) { examples.Add(sharedExample); if (!sharedExample.IsNewLine) { validExamples.Add(sharedExample); } } } var i = 0; foreach (var actionDependentFeature in actionDependentFeatures) { var adfExample = actionDependentFeatureSerializer.Serialize(actionDependentFeature, index != null && i == index ? label : null); Contract.Assert(adfExample != null); examples.Add(adfExample); if (!adfExample.IsNewLine) { validExamples.Add(adfExample); validActionDependentFeatures.Add(new ActionDependentFeature(i, actionDependentFeature)); } else { emptyActionDependentFeatures.Add(new ActionDependentFeature(i, actionDependentFeature)); } i++; } if (validActionDependentFeatures.Count == 0) return; // signal we're finished using an empty example emptyExample = vw.GetOrCreateNativeExample(); validExamples.Add(emptyExample); emptyExample.MakeEmpty(vw); predictOrLearn(validExamples, validActionDependentFeatures, emptyActionDependentFeatures); } finally { if (emptyExample != null) emptyExample.Dispose(); // dispose examples // Note: must not dispose examples before final example // as the learning algorithm (such as cbf) keeps a reference // to the example foreach (var e in examples) e.Dispose(); } } /// /// Simplify learning of examples with action dependent features. /// public static void Learn( VowpalWabbit vw, VowpalWabbitSingleExampleSerializer serializer, VowpalWabbitSingleExampleSerializer actionDependentFeatureSerializer, TExample example, IReadOnlyCollection actionDependentFeatures, int index, ILabel label) { Contract.Requires(vw != null); Contract.Requires(actionDependentFeatureSerializer != null); Contract.Requires(example != null); Contract.Requires(actionDependentFeatures != null); Contract.Requires(index >= 0); Contract.Requires(label != null); Execute( vw, serializer, actionDependentFeatureSerializer, example, actionDependentFeatures, (examples, _, __) => { foreach (var ex in examples) { vw.Learn(ex); } }, index, label); } /// /// Simplify learning of examples with action dependent features. /// /// The type of the user example. /// The type of the user action dependent features. /// The vw instance. /// The serializer for . /// The serializer for . /// The user example. /// The action dependent features. /// The index of action dependent feature to label. /// The label for the selected action dependent feature. /// An ranked subset of predicted actions. public static ActionDependentFeature[] LearnAndPredict( VowpalWabbit vw, VowpalWabbitSingleExampleSerializer serializer, VowpalWabbitSingleExampleSerializer actionDependentFeatureSerializer, TExample example, IReadOnlyCollection actionDependentFeatures, int index, ILabel label) { Contract.Requires(vw != null); Contract.Requires(actionDependentFeatureSerializer != null); Contract.Requires(example != null); Contract.Requires(actionDependentFeatures != null); Contract.Requires(index >= 0); Contract.Requires(label != null); ActionDependentFeature[] predictions = null; Execute( vw, serializer, actionDependentFeatureSerializer, example, actionDependentFeatures, (examples, validActionDependentFeatures, emptyActionDependentFeatures) => { foreach (var ex in examples) { vw.Learn(ex); } predictions = VowpalWabbitMultiLine.GetPrediction(vw, examples, validActionDependentFeatures, emptyActionDependentFeatures); }, index, label); // default to the input list return predictions ?? actionDependentFeatures.Select((o, i) => new ActionDependentFeature(i, o)).ToArray(); } /// /// Simplify prediction of examples with action dependent features. /// /// The type of the user example. /// The type of the user action dependent features. /// The vw instance. /// The serializer for . /// The serializer for . /// The user example. /// The action dependent features. /// The index of action dependent feature to label. /// The label for the selected action dependent feature. /// An ranked subset of predicted actions. public static ActionDependentFeature[] Predict( VowpalWabbit vw, VowpalWabbitSingleExampleSerializer serializer, VowpalWabbitSingleExampleSerializer actionDependentFeatureSerializer, TExample example, IReadOnlyCollection actionDependentFeatures, int? index = null, ILabel label = null) { Contract.Requires(vw != null); Contract.Requires(actionDependentFeatureSerializer != null); Contract.Requires(example != null); Contract.Requires(actionDependentFeatures != null); ActionDependentFeature[] predictions = null; Execute( vw, serializer, actionDependentFeatureSerializer, example, actionDependentFeatures, (examples, validActionDependentFeatures, emptyActionDependentFeatures) => { foreach (var ex in examples) { vw.Predict(ex); } predictions = VowpalWabbitMultiLine.GetPrediction(vw, examples, validActionDependentFeatures, emptyActionDependentFeatures); }, index, label); // default to the input list return predictions ?? actionDependentFeatures.Select((o, i) => new ActionDependentFeature(i, o)).ToArray(); } /// /// Extracts the prediction, orders the action depdendent feature objects accordingly and appends the /// action dependent feature objcts that did produce empty examples at the end. /// /// The action dependent feature type. /// The Vowpal Wabbit instance.> /// The list of examples. /// The list of non-empty action dependent feature objects. /// The list of empty action dependent feature objects. /// Returns the ranked list of action dependent features. public static ActionDependentFeature[] GetPrediction( VowpalWabbit vw, IReadOnlyList examples, IReadOnlyList> validActionDependentFeatures, IReadOnlyList> emptyActionDependentFeatures) { // Since the prediction result is stored in the first example // and we'll have to get an actual VowpalWabbitExampt var firstExample = examples.FirstOrDefault(); if (firstExample == null) { return null; } ActionDependentFeature[] result; int i = 0; var values = firstExample.GetPrediction(vw, VowpalWabbitPredictionType.Dynamic); var actionScores = values as ActionScore[]; if (actionScores != null) { if (actionScores.Length != validActionDependentFeatures.Count) throw new InvalidOperationException("Number of predictions returned unequal number of examples fed"); result = new ActionDependentFeature[validActionDependentFeatures.Count + emptyActionDependentFeatures.Count]; foreach (var index in actionScores) { result[i] = validActionDependentFeatures[(int)index.Action]; result[i].Probability = index.Score; i++; } } else { var multilabel = values as int[]; if (multilabel != null) { if (multilabel.Length != validActionDependentFeatures.Count) throw new InvalidOperationException("Number of predictions returned unequal number of examples fed"); result = new ActionDependentFeature[validActionDependentFeatures.Count + emptyActionDependentFeatures.Count]; foreach (var index in multilabel) result[i++] = validActionDependentFeatures[index]; result[0].Probability = 1f; } else throw new NotSupportedException("Unsupported return type: " + values.GetType()); } // append invalid ones at the end foreach (var f in emptyActionDependentFeatures) result[i++] = f; return result; } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/VowpalWabbitMultiLineExampleCollection.cs000066400000000000000000000163151323713302100264230ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Diagnostics.Contracts; using System.Linq; using VW.Labels; namespace VW { /// /// Result for multiline examples. /// public sealed class VowpalWabbitMultiLineExampleCollection : VowpalWabbitExampleCollection { private readonly ulong numberOfFeatures; /// /// Initializes a new instance of the class. /// public VowpalWabbitMultiLineExampleCollection(VowpalWabbit vw, VowpalWabbitExample shared, VowpalWabbitExample[] examples) : base(vw) { Contract.Requires(examples != null); this.SharedExample = shared; this.Examples = examples; if (shared != null) numberOfFeatures += shared.NumberOfFeatures; foreach (var e in examples) if (e != null) numberOfFeatures += e.NumberOfFeatures; } /// /// Single example or the shared /// public VowpalWabbitExample SharedExample { get; private set; } /// /// The multi-line examples /// public VowpalWabbitExample[] Examples { get; private set; } /// /// The number of feature this example holds. /// public override ulong NumberOfFeatures { get { return this.numberOfFeatures; } } /// /// Calls learn or predict for the set of examples. Does required filtering of potential new line examples. /// private TPrediction Execute(VowpalWabbit vw, Action predictOrLearn, IVowpalWabbitPredictionFactory predictionFactory = null) { Contract.Requires(predictOrLearn != null); // firstExample will contain prediction result VowpalWabbitExample firstExample = null; VowpalWabbitExample empty = null; try { if (this.SharedExample != null && !this.SharedExample.IsNewLine) { predictOrLearn(this.SharedExample); firstExample = this.SharedExample; } foreach (var ex in this.Examples) { if (!ex.IsNewLine) { predictOrLearn(ex); if (firstExample == null) firstExample = ex; } } // signal end-of-block empty = vw.GetOrCreateNativeExample(); empty.MakeEmpty(vw); predictOrLearn(empty); return predictionFactory != null ? firstExample.GetPrediction(vw, predictionFactory) : default(TPrediction); } finally { if (empty != null) empty.Dispose(); } } /// /// Learns from these examples. /// protected override void LearnInternal(VowpalWabbit vw) { // unfortunately can't specify this.Execute(vw, ex => vw.Learn(ex)); } /// /// Learn from these examples and returns the current prediction for it. /// /// The prediction type. /// The prediction factory to be used. See . /// The VW instance that should be used for learning. /// The prediction for the this example. protected override TPrediction LearnInternal(IVowpalWabbitPredictionFactory predictionFactory, VowpalWabbit vw) { return this.Execute(vw, ex => vw.Learn(ex), predictionFactory); } /// /// Predicts for these examples. /// protected override void PredictInternal(VowpalWabbit vw) { // unfortunately can't specify this.Execute(vw, ex => vw.Predict(ex)); } /// /// Predicts for these examples and returns the current prediction for it. /// /// The prediction type. /// The prediction factory to be used. See . /// The native VW instance. /// The prediction for the this example. protected override TPrediction PredictInternal(IVowpalWabbitPredictionFactory predictionFactory, VowpalWabbit vw) { return this.Execute(vw, ex => vw.Predict(ex), predictionFactory); } /// /// The optional string version of the example. /// public override string VowpalWabbitString { get { var str = new List(); if (this.SharedExample != null) str.Add(this.SharedExample.VowpalWabbitString); str.AddRange(this.Examples.Select(e => e.VowpalWabbitString)); // filter empty example return string.Join("\n", str.Where(s => !string.IsNullOrWhiteSpace(s))); } } /// /// All labels this example holds. /// public override IEnumerable Labels { get { return this.Examples.Select(e => e.Label); } } /// /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources. /// public override void Dispose() { this.Dispose(true); GC.SuppressFinalize(this); } private void Dispose(bool disposing) { if (disposing) { if (this.SharedExample != null) { this.SharedExample.Dispose(); this.SharedExample = null; } if (this.Examples != null) { foreach (var ex in this.Examples) if (ex != null) ex.Dispose(); this.Examples = null; } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/VowpalWabbitMultiworldTesting.cs000066400000000000000000000122311323713302100246620ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.ComponentModel; using System.IO; using System.Linq; using VW.Labels; using VW.Serializer.Attributes; namespace VW { /// /// A wrapper for --multiworld_testing mode. /// public sealed class VowpalWabbitMultiworldTesting : IDisposable { private VowpalWabbit vw; /// /// /// /// Optional model to see multiworld testing public VowpalWabbitMultiworldTesting(Stream vwModel = null) { var settings = vwModel == null ? new VowpalWabbitSettings("--multiworld_test f") : new VowpalWabbitSettings { ModelStream = vwModel }; this.vw = new VowpalWabbit(settings); } /// /// Evaluates and x constants policies w.r.t. to . /// /// The learned action. /// The number constant policies to be evaluated. /// The label. /// public PoliciesPerformance Evaluate(uint learnedAction, int numActions, ContextualBanditLabel label) { return new PoliciesPerformance( this.vw.Learn( new LearnedVsConstantPolicy(learnedAction, numActions), label, VowpalWabbitPredictionType.Scalars)); } /// /// The assocated VW instance. /// public VowpalWabbit Native { get { return this.vw.Native; } } /// /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources. /// public void Dispose() { this.Dispose(true); GC.SuppressFinalize(this); } private void Dispose(bool disposing) { if (disposing) { if (this.vw != null) { this.vw.Dispose(); this.vw = null; } } } /// /// Contains evaluation result for constant policies and the currently learned policy. /// public sealed class PoliciesPerformance { private float[] data; /// /// Initializes a new instance. /// /// The performance data. Learned policy is at position 0. public PoliciesPerformance(float[] data) { this.data = data; } /// /// The performance of the learned policy. /// public float LearnedPolicy { get { return this.data[0]; } } /// /// The number of constant policies evaluated. /// public int NumConstantPolicies { get { return this.data.Length - 1; } } /// /// The performance of each constant policy. /// public IEnumerable ConstantPolicies { get { return this.data.Skip(1); } } } /// /// Must be public for the serializer to work with it. /// [EditorBrowsableAttribute(EditorBrowsableState.Never)] public sealed class LearnedVsConstantPolicy { private uint learnedAction; private int numConstantActions; internal LearnedVsConstantPolicy(uint learnedAction, int numConstantActions) { this.learnedAction = learnedAction; this.numConstantActions = numConstantActions; } /// /// The constant policies actions. /// [Feature(FeatureGroup = 'f')] public IEnumerable Actions { get { yield return learnedAction; for (uint i = 0; i < this.numConstantActions; i++) { yield return i + 1; } } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/VowpalWabbitSerializationException.cs000066400000000000000000000026321323713302100256620ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using VW.Serializer.Intermediate; namespace VW { /// /// Exception thrown while serialization. /// public class VowpalWabbitSerializationException : Exception { /// /// Constructs new exception /// public VowpalWabbitSerializationException(string message, Exception innerException, Namespace ns, Feature feature) : base($"{message}. Namespace: {ns.Name}. Feature: {feature.Name}", innerException) { this.Namespace = ns; this.Feature = feature; } /// /// The related namespace for this exception. /// public Namespace Namespace { get; private set; } /// /// The related feature for this feature. /// public Feature Feature { get; private set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/VowpalWabbitSingleLineExampleCollection.cs000066400000000000000000000107661323713302100265560ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Diagnostics.Contracts; using System.Linq; using System.Text; using System.Threading.Tasks; using VW.Labels; namespace VW { /// /// Result for a single example. /// public sealed class VowpalWabbitSingleLineExampleCollection : VowpalWabbitExampleCollection { /// /// Initializes a new instance of the class. /// public VowpalWabbitSingleLineExampleCollection(VowpalWabbit vw, VowpalWabbitExample example) : base(vw) { Contract.Requires(example != null); this.Example = example; } /// /// Single example or the shared /// public VowpalWabbitExample Example { get; private set; } /// /// Learns from this example. /// protected override void LearnInternal(VowpalWabbit vw) { vw.Learn(this.Example); } /// /// Learn from this example and returns the current prediction for it. /// /// The prediction type. /// The prediction factory to be used. See . /// The VW native instance. /// The prediction for the this example. protected override TPrediction LearnInternal(IVowpalWabbitPredictionFactory predictionFactory, VowpalWabbit vw) { return vw.Learn(this.Example, predictionFactory); } /// /// Predicts for this example. /// protected override void PredictInternal(VowpalWabbit vw) { vw.Predict(this.Example); } /// /// Predicts for this example and returns the current prediction for it. /// /// The prediction type. /// The prediction factory to be used. See . /// The VW instance that should be used for prediction. /// The prediction for the this example. protected override TPrediction PredictInternal(IVowpalWabbitPredictionFactory predictionFactory, VowpalWabbit vw) { return vw.Predict(this.Example, predictionFactory); } /// /// The optional string version of the example. /// public override string VowpalWabbitString { get { return this.Example.VowpalWabbitString; } } /// /// The number of feature this example holds. /// public override ulong NumberOfFeatures { get { return this.Example.NumberOfFeatures; } } /// /// All labels this example holds. /// public override IEnumerable Labels { get { yield return this.Example.Label; } } /// /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources. /// public override void Dispose() { this.Dispose(true); GC.SuppressFinalize(this); } private void Dispose(bool disposing) { if (disposing) { if (this.Example != null) { this.Example.Dispose(); this.Example = null; } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/VowpalWabbitSweep.cs000066400000000000000000000223061323713302100222510ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Concurrent; using System.Collections.Generic; using System.Diagnostics.Contracts; using System.Linq; using System.Text; using System.Threading.Tasks; using VW.Labels; using VW.Serializer; namespace VW { /// /// Sweeping wrapper for multiline examples. Designed to re-use allocated examples /// across multiple Vowpal Wabbit instances. So far plain parallelization yielded /// faster training times at least on a 20 core machine. /// /// User example type. /// Action dependent feature type. public class VowpalWabbitSweep : IDisposable { private const int NumberOfVWInstancesSharingExamples = 1; private VowpalWabbit[] vws; private List settings; private VowpalWabbitSingleExampleSerializer[] serializers; private VowpalWabbitSingleExampleSerializer[] actionDependentFeatureSerializers; /// /// Initializes a new instance. /// /// The list of settings to be used. public VowpalWabbitSweep(List settings) { if (settings == null || settings.Count == 0) throw new ArgumentException("settings"); Contract.EndContractBlock(); // TODO: check that the sweeps are not across incompatible options. this.settings = settings; this.vws = settings.Select(setting => new VowpalWabbit(setting)).ToArray(); var diffs = this.vws.Skip(1).Select(vw => vw.AreFeaturesCompatible(this.vws[0])).Where(e => e != null).ToList(); if (diffs.Count > 0) throw new ArgumentException("Feature settings are not compatible for sweeping: " + string.Join(",", diffs)); this.serializers = this.vws.Select(vw => (VowpalWabbitSingleExampleSerializer)VowpalWabbitSerializerFactory.CreateSerializer(vw.Settings).Create(vw)) .ToArray(); this.actionDependentFeatureSerializers = this.vws.Select(vw => (VowpalWabbitSingleExampleSerializer)VowpalWabbitSerializerFactory.CreateSerializer(vw.Settings).Create(vw)) .ToArray(); } /// /// The internally allocated VW instances. /// public VowpalWabbit[] VowpalWabbits { get { return this.vws; } } /// /// Creates a partioner used for parallel scenarios. /// /// An ordered partitioner. public OrderablePartitioner> CreatePartitioner() { return Partitioner.Create(0, this.vws.Length, Math.Min(this.vws.Length, NumberOfVWInstancesSharingExamples)); } /// /// Learn from the given example and return the current prediction for it. /// /// The shared example. /// Instance number to start from. /// Instance number to end at. /// The action dependent features. /// The index of the example to learn within . /// The label for the example to learn. public void Learn(int fromInclusive, int toExclusive, TExample example, IReadOnlyCollection actionDependentFeatures, int index, ILabel label) { if (fromInclusive != toExclusive - 1) throw new ArgumentException("fromInclusive"); VowpalWabbitMultiLine.Execute(this.vws[fromInclusive], this.serializers[fromInclusive], this.actionDependentFeatureSerializers[fromInclusive], example, actionDependentFeatures, (examples, _, __) => { for (int i = fromInclusive; i < toExclusive; i++) { foreach (var ex in examples) { this.vws[i].Learn(ex); } } }, index, label); } /// /// Prediction sweep. /// /// /// /// /// /// /// /// public TActionDependentFeature[][] Predict(int fromInclusive, int toExclusive, TExample example, IReadOnlyCollection actionDependentFeatures, int index, ILabel label) { if (fromInclusive != toExclusive - 1) throw new ArgumentException("fromInclusive"); var result = new TActionDependentFeature[toExclusive - fromInclusive][]; VowpalWabbitMultiLine.Execute(this.vws[fromInclusive], this.serializers[fromInclusive], this.actionDependentFeatureSerializers[fromInclusive], example, actionDependentFeatures, (examples, validActionDependentFeatures, emptyActionDependentFeatures) => { for (int i = fromInclusive; i < toExclusive; i++) { // feed all examples for this block foreach (var ex in examples) { this.vws[i].Predict(ex); } result[i - fromInclusive] = VowpalWabbitMultiLine.GetPrediction(this.vws[i], examples, validActionDependentFeatures, emptyActionDependentFeatures) .Select(p => p.Feature).ToArray(); } }, index, label); return result; } /// /// Save all models with the given prfix. /// /// /// public List SaveModels(string modelPrefix)         {             return this.vws.Select((vw, i) =>             {                 var modelName = modelPrefix + "-" + i;                 vw.SaveModel(modelName);                 return modelName;             })             .ToList();         }  /// /// Reload all models. /// public void Reload()         {             foreach (var vw in this.vws)             {                 vw.Reload();             }         }  /// /// Executes the given action on each VW instance. /// /// The action to execute. public void Execute(Action, VowpalWabbitSingleExampleSerializer, int> action)         {             Parallel.For(                 0, this.vws.Length,                 new ParallelOptions { MaxDegreeOfParallelism = Environment.ProcessorCount / 2 },                 i => action(this.vws[i], this.serializers[i], this.actionDependentFeatureSerializers[i], i));         }  /// /// Dispose resources. /// public void Dispose() { this.Dispose(true); GC.SuppressFinalize(this); } private void Dispose(bool disposing) { if (disposing) { if (this.vws != null) { foreach (var vw in this.vws) { vw.Dispose(); } this.vws = null; } if (this.serializers != null) { foreach (var s in this.serializers) { s.Dispose(); } this.serializers = null; } if (this.actionDependentFeatureSerializers != null) { foreach (var s in this.actionDependentFeatureSerializers) { s.Dispose(); } this.actionDependentFeatureSerializers = null; } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/VowpalWabbitThreadedPrediction.cs000066400000000000000000000143301323713302100247250ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using VW.Serializer; namespace VW { /// /// Enables multi-threaded prediction by utilizing a pool of instances. /// public sealed class VowpalWabbitThreadedPrediction : VowpalWabbitThreadedPredictionBase { /// /// Initializes a new instance of . /// /// Decision Service client library needs default constructor. public VowpalWabbitThreadedPrediction() { } /// /// Initializes a new instance of . /// /// The model used by each pool instance. public VowpalWabbitThreadedPrediction(VowpalWabbitModel model) : base(model) { } /// /// Initializes a new instance of . /// /// The settings used by each pool instance. public VowpalWabbitThreadedPrediction(VowpalWabbitSettings settings) : base(settings) { } /// /// Returns the same instance as no wrapping is required. /// /// The wrapped vw instance. protected override VowpalWabbit InternalCreate(VowpalWabbit vw) { return vw; } } /// /// Enables multi-threaded prediction by utilizing a pool of instances. /// /// The type use for providing data to VW using the serializer infrastructure. public sealed class VowpalWabbitThreadedPrediction : VowpalWabbitThreadedPredictionBase> { /// /// Initializes a new instance of . /// /// Decision Service client library needs default constructor. public VowpalWabbitThreadedPrediction() { } /// /// Initializes a new instance of . /// /// The model used by each pool instance. /// Optional pre-compiled serializer. public VowpalWabbitThreadedPrediction(VowpalWabbitModel model, IVowpalWabbitSerializerCompiler compiledSerializer = null) : base(model) { this.CompiledSerializer = compiledSerializer; } /// /// Initializes a new instance of . /// /// The settings used by each pool instance. public VowpalWabbitThreadedPrediction(VowpalWabbitSettings settings) : base(settings) { } /// /// The Serializer used to marshal examples. /// public IVowpalWabbitSerializerCompiler CompiledSerializer { get; private set; } /// /// Creates a new instance of . /// /// The wrapped vw instance. protected override VowpalWabbit InternalCreate(VowpalWabbit vw) { if (this.CompiledSerializer == null) this.CompiledSerializer = VowpalWabbitSerializerFactory.CreateSerializer(vw.Settings); return new VowpalWabbit(vw, this.CompiledSerializer); } } /// /// Enables multi-threaded prediction by utilizing a pool of instances. /// /// The type use for providing data to VW using the serializer infrastructure. /// The type use for providing action dependent data to VW using the serializer infrastructure. public sealed class VowpalWabbitThreadedPrediction : VowpalWabbitThreadedPredictionBase> { /// /// Initializes a new instance of . /// /// Decision Service client library needs default constructor. public VowpalWabbitThreadedPrediction() { } /// /// Initializes a new instance of . /// /// The model used by each pool instance. public VowpalWabbitThreadedPrediction(VowpalWabbitModel model) : base(model) { } /// /// Initializes a new instance of . /// /// The settings used by each pool instance. public VowpalWabbitThreadedPrediction(VowpalWabbitSettings settings) : base(settings) { } /// /// Creates a new instance of . /// /// The wrapped vw instance. protected override VowpalWabbit InternalCreate(VowpalWabbit vw) { return new VowpalWabbit(vw); } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/VowpalWabbitThreadedPredictionBase.cs000066400000000000000000000105771323713302100255310ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Diagnostics; using System.Diagnostics.Contracts; namespace VW { /// /// Enables multi-threaded prediction by utilizing a pool of instances. /// /// The VowpalWabbit wrapper type used. public abstract class VowpalWabbitThreadedPredictionBase : IDisposable where TVowpalWabbit : class, IDisposable { /// /// The pool of potentially wrapped VW instances. /// private ObjectPool vwPool; private VowpalWabbitSettings settings; /// /// Initializes a new instance of the class. /// /// The initial model to use. protected VowpalWabbitThreadedPredictionBase(VowpalWabbitModel model = null) : this(new VowpalWabbitSettings() { Model = model }) { } /// /// Initializes a new instance of the class. /// /// The initial settings to use. protected VowpalWabbitThreadedPredictionBase(VowpalWabbitSettings settings) { this.settings = settings; this.vwPool = new ObjectPool( ObjectFactory.Create( settings.Model, m => { if (m == null) return default(TVowpalWabbit); return CreateVowpalWabbitChild(m); })); } private TVowpalWabbit CreateVowpalWabbitChild(VowpalWabbitModel model) { var newSettings = (VowpalWabbitSettings)this.settings.Clone(); newSettings.Model = model; var vw = new VowpalWabbit(newSettings); return this.InternalCreate(vw); } /// /// Implementors create new VW wrapper instances. /// /// The native VW instance. /// The new VW wrapper instance. protected abstract TVowpalWabbit InternalCreate(VowpalWabbit vw); /// /// Updates the model used for prediction in a thread-safe manner. /// /// The new model to be used. public void UpdateModel(VowpalWabbitModel model) { this.vwPool.UpdateFactory(ObjectFactory.Create( model, this.CreateVowpalWabbitChild)); } /// /// Gets or creates a new VW wrapper instance. /// /// A ready to use VW wrapper instance. /// can be null if no model was supplied yet. public PooledObject GetOrCreate() { Contract.Ensures(Contract.Result>() != null); return this.vwPool.GetOrCreate(); } /// /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources. /// public void Dispose() { this.Dispose(true); GC.SuppressFinalize(this); } private void Dispose(bool disposing) { if (disposing) { if (this.vwPool != null) { this.vwPool.Dispose(); this.vwPool = null; } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs/cs.csproj000066400000000000000000000325641323713302100201530ustar00rootroot00000000000000 Debug AnyCPU {E4E962AE-7056-4EB0-A8C5-8DC824A4B068} Library Properties VW VowpalWabbit v4.5.2 512 true ..\..\vowpalwabbit\ 0 true $(SolutionDir)\x64\Debug\ TRACE;DEBUG full x64 prompt MinimumRecommendedRules.ruleset ..\..\vowpalwabbit\x64\Debug\VowpalWabbit.XML True False True False False True True True True True True True True True False True False True False False False False True False True True True False False True False False True Full %28none%29 0 true $(SolutionDir)\x64\Release\ TRACE true pdbonly x64 prompt MinimumRecommendedRules.ruleset ..\..\vowpalwabbit\x64\Release\VowpalWabbit.XML true true ..\vw_key.snk true $(SolutionDir)\x64\DebugLeakCheck\ TRACE;DEBUG true $(SolutionDir)\x64\DebugLeakCheck\VowpalWabbit.XML full x64 prompt MinimumRecommendedRules.ruleset True True VowpalWabbitDefaultMarshallerExt.tt {85e55ae0-3784-4968-9271-c81af560e1c1} vw_clr {e621e022-c1f8-433f-905a-ab9a3de072b7} vw_common Designer TextTemplatingFileGenerator VowpalWabbitDefaultMarshallerExt.cs This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. vowpal-wabbit-8.5.0.dfsg1/cs/cs/cs.nuspec000066400000000000000000000040231323713302100201350ustar00rootroot00000000000000 Vowpal Wabbit VowpalWabbit vw vowpal wabbit langford ml machine learning John Langford et al $version$ https://github.com/JohnLangford/vowpal_wabbit/wiki/C%23-Binding https://github.com/JohnLangford/vowpal_wabbit/blob/master/LICENSE false Official Vowpal Wabbit library including C# interface Copyright (C) Microsoft Corp 2012-2016, Yahoo! Inc. 2007-2012, and many individual contributors. All rights reserved. vowpal-wabbit-8.5.0.dfsg1/cs/cs/packages.config000066400000000000000000000005761323713302100212670ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/cs/web.config.transform000066400000000000000000000005421323713302100222710ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/cs_console/000077500000000000000000000000001323713302100200345ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/cs_console/App.config000066400000000000000000000010341323713302100217410ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/cs_console/Program.cs000066400000000000000000000103561323713302100217770ustar00rootroot00000000000000using Newtonsoft.Json; using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Reflection; using System.Text; using System.Threading.Tasks; using VW; namespace cs_vw { public class Program { enum FileMode { JsonArray, JsonNewLine } private static FileMode DetectedFileMode(string file) { // auto-detect if it's line separated or array based using (var reader = new StreamReader(file)) { var buffer = new char[32]; int readChars; while ((readChars = reader.Read(buffer, 0, buffer.Length)) > 0) { for (int i = 0; i < readChars; i++) { var c = buffer[i]; switch (c) { case ' ': case '\t': case '\n': case '\r': continue; case '[': return FileMode.JsonArray; case '{': return FileMode.JsonNewLine; default: throw new ArgumentException("Invalid character: " + c); } } } } throw new ArgumentException("Empty file"); } public static void Main(string[] args) { // first argument needs to end with .json if (args.Length == 0) { Console.Error.WriteLine( "Usage: {0} ...", Path.GetFileName(Assembly.GetExecutingAssembly().Location)); return; } try { var json = args[0]; var vwArguments = string.Join(" ", args.Skip(1)); var fileMode = DetectedFileMode(json); using (var vw = new VowpalWabbitJson(vwArguments)) { switch (fileMode) { case FileMode.JsonArray: using (var reader = new JsonTextReader(new StreamReader(json))) { if (!reader.Read()) return; if (reader.TokenType != JsonToken.StartArray) return; while (reader.Read()) { switch (reader.TokenType) { case JsonToken.StartObject: vw.Learn(reader); break; case JsonToken.EndObject: // skip break; case JsonToken.EndArray: // end reading return; } } } break; case FileMode.JsonNewLine: using (var reader = new StreamReader(json)) { string line; while ((line = reader.ReadLine()) != null) { if (string.IsNullOrWhiteSpace(line)) continue; vw.Learn(line); } } break; } } } catch (Exception e) { Console.Error.WriteLine("Exception: {0}.\n{1}", e.Message, e.StackTrace); } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs_console/Properties/000077500000000000000000000000001323713302100221705ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/cs_console/Properties/AssemblyInfo.cs000066400000000000000000000022331323713302100251120ustar00rootroot00000000000000//------------------------------------------------------------------------------ // // This code was generated by a tool. // Runtime Version:4.0.30319.42000 // // Changes to this file may cause incorrect behavior and will be lost if // the code is regenerated. // //------------------------------------------------------------------------------ [assembly: System.Reflection.AssemblyTitle("Vowpal Wabbit Console")] [assembly: System.Reflection.AssemblyDescription("Vowpal Wabbit Console")] [assembly: System.Reflection.AssemblyCompany("Microsoft Corp")] [assembly: System.Reflection.AssemblyProduct("Vowpal Wabbit")] [assembly: System.Reflection.AssemblyCopyright("Copyright (C) Microsoft Corp 2012-2016, Yahoo! Inc. 2007-2012, and many individua" + "l contributors. All rights reserved")] [assembly: System.Runtime.InteropServices.ComVisible(false)] [assembly: System.CLSCompliant(false)] [assembly: System.Runtime.InteropServices.Guid("c7c26e42-6d03-4fe5-943c-add2440f1e37")] [assembly: System.Reflection.AssemblyVersion("8.4.0.1")] [assembly: System.Reflection.AssemblyFileVersion("8.4.0.1")] vowpal-wabbit-8.5.0.dfsg1/cs/cs_console/cs_console.csproj000066400000000000000000000132221323713302100234050ustar00rootroot00000000000000 Debug AnyCPU {01A85382-C3E9-480A-86BF-FAFE4AD107A7} Exe Properties VowpalWabbit.Console vw_json v4.5.2 512 $(SolutionDir)\ true ..\vw_key.snk true $(SolutionDir)\x64\Debug\ DEBUG;TRACE full x64 prompt MinimumRecommendedRules.ruleset true $(SolutionDir)\x64\Release\ TRACE true pdbonly x64 prompt MinimumRecommendedRules.ruleset true $(SolutionDir)\packages\Newtonsoft.Json.9.0.1\lib\net45\Newtonsoft.Json.dll True Designer Designer {85e55ae0-3784-4968-9271-c81af560e1c1} vw_clr {e621e022-c1f8-433f-905a-ab9a3de072b7} vw_common {e4e962ae-7056-4eb0-a8c5-8dc824a4b068} cs {9e27fa94-ab34-4736-8427-fb7a2ba90d52} cs_json This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. vowpal-wabbit-8.5.0.dfsg1/cs/cs_console/packages.config000066400000000000000000000003651323713302100230050ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/cs_console/setenv.bat000066400000000000000000000003311323713302100220250ustar00rootroot00000000000000@ECHO OFF SET Path=%Path%;%programfiles%\VowpalWabbit @echo Welcome to Vowpal Wabbit Enviroment @echo. @echo vw.exe ... regular Vowpal Wabbit machine learning @echo vw_json.exe ... JSON ingesting Vowpal Wabbit vowpal-wabbit-8.5.0.dfsg1/cs/cs_json/000077500000000000000000000000001323713302100173435ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/cs_json/Properties/000077500000000000000000000000001323713302100214775ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/cs_json/Properties/AssemblyInfo.cs000066400000000000000000000022251323713302100244220ustar00rootroot00000000000000//------------------------------------------------------------------------------ // // This code was generated by a tool. // Runtime Version:4.0.30319.42000 // // Changes to this file may cause incorrect behavior and will be lost if // the code is regenerated. // //------------------------------------------------------------------------------ [assembly: System.Reflection.AssemblyTitle("Vowpal Wabbit JSON")] [assembly: System.Reflection.AssemblyDescription("Vowpal Wabbit JSON")] [assembly: System.Reflection.AssemblyCompany("Microsoft Corp")] [assembly: System.Reflection.AssemblyProduct("Vowpal Wabbit")] [assembly: System.Reflection.AssemblyCopyright("Copyright (C) Microsoft Corp 2012-2016, Yahoo! Inc. 2007-2012, and many individua" + "l contributors. All rights reserved")] [assembly: System.Runtime.InteropServices.ComVisible(false)] [assembly: System.CLSCompliant(false)] [assembly: System.Runtime.InteropServices.Guid("8a34db14-bac2-474b-8102-be25ca5f2c55")] [assembly: System.Reflection.AssemblyVersion("8.4.0.1")] [assembly: System.Reflection.AssemblyFileVersion("8.4.0.1")] vowpal-wabbit-8.5.0.dfsg1/cs/cs_json/Serializer/000077500000000000000000000000001323713302100214545ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/cs_json/Serializer/IVowpalWabbitJsonConverter.cs000066400000000000000000000017611323713302100272440ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System.Collections.Generic; namespace VW.Serializer { /// /// Optimization interface for JsonConverter holding one or more independently parseable JSON fragments. /// /// /// This avoids string copying. /// public interface IVowpalWabbitJsonConverter { /// /// List of independently parseable JSON fragments. /// IEnumerable JsonFragments(object value); } } vowpal-wabbit-8.5.0.dfsg1/cs/cs_json/Serializer/JsonRawStringConverter.cs000066400000000000000000000041161323713302100264470ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Newtonsoft.Json; using System; using System.Collections.Generic; namespace VW.Serializer { /// /// Custom JSON converter returning the underlying raw json (avoiding object allocation). /// public class JsonRawStringConverter : JsonConverter, IVowpalWabbitJsonConverter { /// /// Supports string only. /// public override bool CanConvert(Type objectType) { return objectType == typeof(string); } /// /// Not implemented. /// public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer) { throw new NotImplementedException(); } /// /// Outputs the string contents as JSON. /// public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer) { var valueString = value as string; if (valueString != null) { writer.WriteRawValue(valueString); return; } serializer.Serialize(writer, value); } /// /// List of independently parseable JSON fragments. /// public IEnumerable JsonFragments(object value) { var valueString = value as string; if (valueString != null) { yield return valueString; yield break; } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs_json/Serializer/JsonRawStringListConverter.cs000066400000000000000000000044311323713302100273030ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Newtonsoft.Json; using System; using System.Collections.Generic; namespace VW.Serializer { /// /// Custom JSON converter returning the underlying raw json (avoiding object allocation). /// public class JsonRawStringListConverter : JsonConverter, IVowpalWabbitJsonConverter { /// /// Supports string only. /// public override bool CanConvert(Type objectType) { return objectType == typeof(List); } /// /// Not implemented. /// public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer) { throw new NotImplementedException(); } /// /// Outputs the string contents as JSON. /// public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer) { var valueStringEnumerable = value as List; if (valueStringEnumerable != null) { writer.WriteStartArray(); foreach (var str in valueStringEnumerable) writer.WriteRawValue(str); writer.WriteEndArray(); return; } serializer.Serialize(writer, value); } /// /// List of independently parseable JSON fragments. /// public IEnumerable JsonFragments(object value) { var valueStringList = value as List; if (valueStringList == null) throw new ArgumentException($"Unsupported type: {value}"); return valueStringList; } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs_json/Serializer/JsonTypeInspector.cs000066400000000000000000000430531323713302100254520ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Newtonsoft.Json; using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Linq.Expressions; using System.Reflection; using System.Text; using VW.Labels; namespace VW.Serializer { /// /// Utility class analyzing compile-time annotation. /// public static class JsonTypeInspector { /// /// The singleton native instance. /// public static readonly ITypeInspector Default = new JsonTypeInspectorImpl(); private sealed class JsonTypeInspectorImpl : ITypeInspector { public Schema CreateSchema(VowpalWabbitSettings settings, Type type) { return JsonTypeInspector.CreateSchema(type, settings.PropertyConfiguration); } } private static readonly Type[] SupportedTypes; private static readonly Type[] DictTypes; static JsonTypeInspector() { var numericElementTypes = new[] { typeof(byte), typeof(sbyte), typeof(Int16), typeof(Int32), typeof(UInt16), typeof(UInt32), typeof(float), typeof(Int64), typeof(UInt64), typeof(double) }; var enumerableType = typeof(IEnumerable).GetGenericTypeDefinition(); var dictType = typeof(IDictionary).GetGenericTypeDefinition(); SupportedTypes = new[] { typeof(char), typeof(bool), typeof(string), typeof(double), typeof(float), typeof(byte), typeof(decimal), typeof(UInt16), typeof(UInt32), typeof(UInt64), typeof(Int16), typeof(Int32), typeof(Int64), } .Union(numericElementTypes.Select(valueType => enumerableType.MakeGenericType(valueType))) .Union(numericElementTypes.Select(valueType => valueType.MakeArrayType())) .ToArray(); DictTypes = numericElementTypes .SelectMany(valueType => new[] { dictType.MakeGenericType(typeof(string), valueType), dictType.MakeGenericType(typeof(string), valueType.MakeArrayType()), }) .ToArray(); } private static bool IsTypeSupported(Type type) { return SupportedTypes.Any(t => t.IsAssignableFrom(type)); } private static bool IsDictType(Type type) { return DictTypes.Any(t => t.IsAssignableFrom(type)); } /// /// Extract the JSON.NET from the type. Defaults to . /// /// is not supported. private static MemberSerialization GetMemberSerialiation(Type type) { var jsonObjectAttr = (JsonObjectAttribute)type.GetCustomAttributes(typeof(JsonObjectAttribute), true).FirstOrDefault(); if (jsonObjectAttr == null) return MemberSerialization.OptOut; if (jsonObjectAttr.MemberSerialization == MemberSerialization.Fields) throw new ArgumentException("MemberSerialization.Fields is set on type " + type + " and is not supported"); return jsonObjectAttr.MemberSerialization; } private static Func CreateValueExpressionFactory(PropertyInfo namespacePropertyInfo, PropertyInfo featurePropertyInfo) { Func baseExpression = v => namespacePropertyInfo == null ? v : // CODE example Expression.Property(v, namespacePropertyInfo); // CODE example.NamespaceProperty var attr = featurePropertyInfo.GetCustomAttribute(typeof(JsonConverterAttribute), true) as JsonConverterAttribute; if (attr == null) // CODE example.FeatureProperty or example.NamespaceProperty.FeatureProperty return example => Expression.Property(baseExpression(example), featurePropertyInfo); // validate var converterCtor = attr.ConverterParameters == null ? attr.ConverterType.GetConstructor(Type.EmptyTypes) : attr.ConverterType.GetConstructor(attr.ConverterParameters.Select(o => o.GetType()).ToArray()); if (converterCtor == null) throw new ArgumentException($"Unable to find constructor for converter '{attr.ConverterType}' for '{featurePropertyInfo.Name}'"); var jsonConverter = converterCtor.Invoke(attr.ConverterParameters) as JsonConverter; if (jsonConverter == null) throw new ArgumentException($"JsonConverter '{attr.ConverterType}' for '{featurePropertyInfo.Name}' is not of type JsonConverter"); if (!jsonConverter.CanConvert(featurePropertyInfo.PropertyType)) throw new ArgumentException($"JsonConverter '{attr.ConverterType}' for '{featurePropertyInfo.Name}' does not support property type '{featurePropertyInfo.PropertyType}'"); // CODE: new JsonConverter*(arg1, arg2,...) var converterExpression = attr.ConverterParameters == null ? Expression.New(converterCtor) : Expression.New(converterCtor, attr.ConverterParameters.Select(o => Expression.Constant(o))); // leverage optimized path var serializableCtor = jsonConverter is IVowpalWabbitJsonConverter ? typeof(VowpalWabbitJsonOptimizedSerializable).GetConstructor(new[] { typeof(object), typeof(IVowpalWabbitJsonConverter) }) : typeof(VowpalWabbitJsonSerializable).GetConstructor(new[] { typeof(object), typeof(JsonConverter) }); // CODE new VowpalWabbitJsonConverter(object, new JsonConverter(...)) return example => Expression.New( serializableCtor, Expression.Property(baseExpression(example), featurePropertyInfo), converterExpression); } /// /// Extracts VW features from given type based on JSON.NET annotation. Basic structure: /// /// { /// _label: { ... }, // SimpleLabel or ContextualBanditLabel /// ns1: { // Complex types denote namespaces. Property name becomes namespace name. /// feature1: 5, // Primitive types denote features /// ... /// }, /// ns2 : { ... }, // another namespace /// feature2: true // Top-level primitive property becomes feature in default namespace. /// } /// internal static Schema CreateSchema(Type type, PropertyConfiguration propertyConfiguration) { var exampleMemberSerialization = GetMemberSerialiation(type); // find all feature properties under namespace properties var namespaceFeatures = from ns in type.GetProperties() // removing any JsonIgnore properties where !ns.GetCustomAttributes(typeof(JsonIgnoreAttribute), true).Any() let nsAttr = (JsonPropertyAttribute)ns.GetCustomAttributes(typeof(JsonPropertyAttribute), true).FirstOrDefault() let nsIsMarkedWithJsonConverter = ns.GetCustomAttribute(typeof(JsonConverterAttribute), true) is JsonConverterAttribute where !IsDictType(ns.PropertyType) && !IsTypeSupported(ns.PropertyType) && !nsIsMarkedWithJsonConverter && // model OptIn/OptOut (exampleMemberSerialization == MemberSerialization.OptOut || (exampleMemberSerialization == MemberSerialization.OptIn && nsAttr != null)) let namespaceRawValue = nsAttr != null && nsAttr.PropertyName != null ? nsAttr.PropertyName : ns.Name // filter all aux properties where !namespaceRawValue.StartsWith(propertyConfiguration.FeatureIgnorePrefix, StringComparison.Ordinal) let featureGroup = namespaceRawValue[0] let namespaceValue = namespaceRawValue.Length > 1 ? namespaceRawValue.Substring(1) : null let namespaceMemberSerialization = GetMemberSerialiation(ns.PropertyType) from p in ns.PropertyType.GetProperties() // removing any JsonIgnore properties where !p.GetCustomAttributes(typeof(JsonIgnoreAttribute), true).Any() let attr = (JsonPropertyAttribute)p.GetCustomAttributes(typeof(JsonPropertyAttribute), true).FirstOrDefault() let isMarkedWithJsonConverter = p.GetCustomAttribute(typeof(JsonConverterAttribute), true) is JsonConverterAttribute where (IsTypeSupported(p.PropertyType) || isMarkedWithJsonConverter) && // model OptIn/OptOut (exampleMemberSerialization == MemberSerialization.OptOut || (exampleMemberSerialization == MemberSerialization.OptIn && attr != null)) let name = attr != null && attr.PropertyName != null ? attr.PropertyName : p.Name let isTextProperty = name == propertyConfiguration.TextProperty // filter all aux properties where isTextProperty || !name.StartsWith(propertyConfiguration.FeatureIgnorePrefix, StringComparison.Ordinal) select new FeatureExpression( featureType: isMarkedWithJsonConverter ? typeof(VowpalWabbitJsonSerializable) : p.PropertyType, name: name, // CODE example.NamespaceProperty.FeatureProperty valueExpressionFactory: CreateValueExpressionFactory(ns, p), // Note: default to string escaping stringProcessing: isTextProperty ? StringProcessing.Split : StringProcessing.EscapeAndIncludeName, // CODE example != null // CODE example.NamespaceProperty != null valueValidExpressionFactories: new List>{ valueExpression => Expression.NotEqual(valueExpression, Expression.Constant(null)), valueExpression => Expression.NotEqual(Expression.Property(valueExpression, ns), Expression.Constant(null)) }, @namespace: namespaceValue, featureGroup: featureGroup); // find all top-level feature properties for the default namespace var defaultNamespaceFeatures = from p in type.GetProperties() // removing any JsonIgnore properties where !p.GetCustomAttributes(typeof(JsonIgnoreAttribute), true).Any() let attr = (JsonPropertyAttribute)p.GetCustomAttributes(typeof(JsonPropertyAttribute), true).FirstOrDefault() where // model OptIn/OptOut (exampleMemberSerialization == MemberSerialization.OptOut || (exampleMemberSerialization == MemberSerialization.OptIn && attr != null)) let name = attr != null && attr.PropertyName != null ? attr.PropertyName : p.Name // filter all aux properties, except for special props where propertyConfiguration.IsSpecialProperty(name) || !name.StartsWith(propertyConfiguration.FeatureIgnorePrefix, StringComparison.Ordinal) // filtering labels for now where name != propertyConfiguration.LabelProperty let isMarkedWithJsonConverter = p.GetCustomAttribute(typeof(JsonConverterAttribute), true) is JsonConverterAttribute where IsTypeSupported(p.PropertyType) || // _multi can be any list type that JSON.NET supports name == propertyConfiguration.MultiProperty || isMarkedWithJsonConverter || // labels must be ILabel or string // Note: from the JSON side they actually can be anything that serializes to the same properties as ILabel implementors (name == propertyConfiguration.LabelProperty && (typeof(ILabel).IsAssignableFrom(p.PropertyType) || p.PropertyType == typeof(string))) select new FeatureExpression( featureType: isMarkedWithJsonConverter ? typeof(VowpalWabbitJsonSerializable) : p.PropertyType, name: name, // CODE example.FeatureProperty valueExpressionFactory: CreateValueExpressionFactory(null, p), // Note: default to string escaping stringProcessing: name == propertyConfiguration.TextProperty ? StringProcessing.Split : StringProcessing.EscapeAndIncludeName, // CODE example != null valueValidExpressionFactories: new List>{ valueExpression => Expression.NotEqual(valueExpression, Expression.Constant(null)) }, @namespace: p.PropertyType.IsArray && name.Length > 1 ? name.Substring(1) : null, featureGroup: p.PropertyType.IsArray && name.Length > 0 ? name[0] : VowpalWabbitConstants.DefaultNamespace); // find all top-level dictionaries var topLevelDictionaries = from p in type.GetProperties() // removing any JsonIgnore properties where !p.GetCustomAttributes(typeof(JsonIgnoreAttribute), true).Any() let attr = (JsonPropertyAttribute)p.GetCustomAttributes(typeof(JsonPropertyAttribute), true).FirstOrDefault() where // model OptIn/OptOut (exampleMemberSerialization == MemberSerialization.OptOut || (exampleMemberSerialization == MemberSerialization.OptIn && attr != null)) where IsDictType(p.PropertyType) let name = attr != null && attr.PropertyName != null ? attr.PropertyName : p.Name let namespaceRawValue = attr != null && attr.PropertyName != null ? attr.PropertyName : p.Name // filter all aux properties where !namespaceRawValue.StartsWith(propertyConfiguration.FeatureIgnorePrefix, StringComparison.Ordinal) let featureGroup = namespaceRawValue[0] let namespaceValue = namespaceRawValue.Length > 1 ? namespaceRawValue.Substring(1) : null select new FeatureExpression( featureType: p.PropertyType, name: name, // CODE example.FeatureProperty valueExpressionFactory: CreateValueExpressionFactory(null, p), // CODE example != null valueValidExpressionFactories: new List> { valueExpression => Expression.NotEqual(valueExpression, Expression.Constant(null)) }, @namespace: namespaceValue, featureGroup: featureGroup); // find label var labelProperties = from p in type.GetProperties() // removing any JsonIgnore properties where !p.GetCustomAttributes(typeof(JsonIgnoreAttribute), true).Any() let attr = (JsonPropertyAttribute)p.GetCustomAttributes(typeof(JsonPropertyAttribute), true).FirstOrDefault() where // model OptIn/OptOut (exampleMemberSerialization == MemberSerialization.OptOut || (exampleMemberSerialization == MemberSerialization.OptIn && attr != null)) let name = attr != null && attr.PropertyName != null ? attr.PropertyName : p.Name // filtering labels for now where name == propertyConfiguration.LabelProperty where // labels must be ILabel or string // Note: from the JSON side they actually can be anything that serializes to the same properties as ILabel implementors (name == propertyConfiguration.LabelProperty && (typeof(ILabel).IsAssignableFrom(p.PropertyType) || p.PropertyType == typeof(string))) select new LabelExpression { LabelType = p.PropertyType, Name = name, // CODE example.Label ValueExpressionFactory = valueExpression => Expression.Property(valueExpression, p), // CODE example != null ValueValidExpressionFactories = new List>{ valueExpression => Expression.NotEqual(valueExpression, Expression.Constant(null)) } }; // TODO: _label_ and _labelIndex is not supported return new Schema { Label = labelProperties.FirstOrDefault(), Features = namespaceFeatures .Union(defaultNamespaceFeatures) .Union(topLevelDictionaries).ToList() }; } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs_json/Serializer/PrefixedJsonReader.cs000066400000000000000000000037041323713302100255320ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Newtonsoft.Json; using System; using System.Collections.Generic; using System.Linq; namespace VW.Serializer { /// /// A Json Reader allowing to prefix data from a wrapped JsonReader. /// internal class PrefixedJsonReader : JsonReader { private JsonReader reader; private Queue> prefix; /// /// Initializes a new instance of . /// /// The reader to be wrapped. /// The JsonTokens to be injected at the beginning of the stream. internal PrefixedJsonReader(JsonReader reader, params Tuple[] prefix) { this.reader = reader; this.prefix = new Queue>(prefix); } /// /// Injects the supplied prefix into the stream. /// /// True if another token is available, false otherwise. public override bool Read() { if (this.prefix.Count > 0) { var t = prefix.Dequeue(); this.SetToken(t.Item1, t.Item2); return true; } if (!this.reader.Read()) return false; this.SetToken(this.reader.TokenType, this.reader.Value); return true; } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs_json/Serializer/VowpalWabbitJsonBuilder.cs000066400000000000000000000766771323713302100265740ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Newtonsoft.Json; using Newtonsoft.Json.Linq; using System; using System.Collections.Generic; using System.Diagnostics.Contracts; using System.Linq; using VW.Labels; using VW.Serializer.Intermediate; namespace VW.Serializer { /// /// Build from JSON following https://github.com/JohnLangford/vowpal_wabbit/wiki/JSON /// public sealed class VowpalWabbitJsonBuilder : IDisposable { /// /// Mapping from properties to types for labels. /// private static readonly Dictionary labelPropertyMapping; private readonly VowpalWabbit vw; private readonly VowpalWabbitDefaultMarshaller defaultMarshaller; private readonly JsonSerializer jsonSerializer; // required for reference resolution private readonly VowpalWabbitJsonSerializer serializer; private readonly VowpalWabbitJsonReferenceResolver referenceResolver; private readonly List namespaceStrings; private JsonReader reader; private bool foundMulti; private JObject labelObject; private ILabel label; private int featureCount; private VowpalWabbitJsonParseState extensionState; private List extensions; static VowpalWabbitJsonBuilder() { // find mapping from property names to types var q = from t in new[] { typeof(SimpleLabel), typeof(ContextualBanditLabel) } from p in t.GetProperties() let jsonProperty = (JsonPropertyAttribute)p.GetCustomAttributes(typeof(JsonPropertyAttribute), true).FirstOrDefault() where jsonProperty != null select new { Type = t, JsonProperty = jsonProperty, Property = p }; labelPropertyMapping = q.ToDictionary( e => (e.JsonProperty.PropertyName ?? e.Property.Name).ToLowerInvariant(), e => e.Type); } /// /// Initializes a new instance of . /// public VowpalWabbitJsonBuilder(IVowpalWabbitExamplePool vwPool, VowpalWabbitDefaultMarshaller defaultMarshaller, JsonSerializer jsonSerializer, int multiIndex = -1) : this(null, vwPool, defaultMarshaller, jsonSerializer, multiIndex) { } /// /// Initializes a new instance of . /// public VowpalWabbitJsonBuilder(VowpalWabbitJsonSerializer serializer, IVowpalWabbitExamplePool vwPool, VowpalWabbitDefaultMarshaller defaultMarshaller, JsonSerializer jsonSerializer, int multiIndex = -1) { Contract.Requires(serializer != null); Contract.Requires(vw != null); Contract.Requires(defaultMarshaller != null); Contract.Requires(jsonSerializer != null); this.extensionState = new VowpalWabbitJsonParseState { JsonBuilder = this, VW = vwPool.Native, MultiIndex = multiIndex }; this.namespaceStrings = new List(); this.foundMulti = false; if (serializer != null) this.referenceResolver = serializer.ReferenceResolver; this.serializer = serializer; this.vw = vwPool.Native; this.defaultMarshaller = defaultMarshaller; this.jsonSerializer = jsonSerializer; this.DefaultNamespaceContext = new VowpalWabbitMarshalContext(this.vw); } // useful for tracking down bugs // private string DefaultNamespaceContextStackTrace; /// /// The marshalling context for the default namespace. Can be modified until . /// public VowpalWabbitMarshalContext DefaultNamespaceContext { get; private set; } /// /// The index the label was assigned to for multi line examples. /// public int LabelIndex { get; private set; } /// /// The label that was deserialized. /// public ILabel Label { get; private set; } /// /// Creates the managed example representation. /// /// Returns the managed example. public VowpalWabbitExample CreateExample() { try { if (this.featureCount == 0) return null; var vwExample = this.DefaultNamespaceContext.ExampleBuilder.CreateExample(); if (this.vw.Settings.EnableStringExampleGeneration) { var str = this.DefaultNamespaceContext.ToString(); if (str.Length > 0) this.namespaceStrings.Insert(0, str); vwExample.VowpalWabbitString = string.Join(" ", this.namespaceStrings); } return vwExample; } finally { // useful for tracking down bugs // this.DefaultNamespaceContextStackTrace = "Create Example" + Environment.StackTrace; this.DefaultNamespaceContext.Dispose(); this.DefaultNamespaceContext = null; } } // re-entering from extension internal void Parse(List path, VowpalWabbitMarshalContext namespaceContext, Namespace ns) { this.featureCount = this.defaultMarshaller.MarshalNamespace(namespaceContext, ns, () => this.ParseProperties(path)) + this.featureCount; } /// /// Parse VW JSON /// public void Parse(JsonReader reader, VowpalWabbitMarshalContext context, Namespace ns, List extensions = null) { this.namespaceStrings.Clear(); this.reader = reader; this.extensions = extensions; // handle the case when the reader is already positioned at JsonToken.StartObject if (reader.TokenType == JsonToken.None && !reader.Read()) return; // don't barf on null values. if (reader.TokenType == JsonToken.Null) return; if (reader.TokenType != JsonToken.StartObject) throw new VowpalWabbitJsonException(this.reader, $"Expected start object. Found '{reader.TokenType}' and value '{reader.Value}' for namespace {ns.Name}"); // re-direct default namespace to the one passed var saveDefaultNamespaceContext = this.DefaultNamespaceContext; try { using (this.DefaultNamespaceContext = new VowpalWabbitMarshalContext(this.vw, context.ExampleBuilder)) { VowpalWabbitJsonParseContext localContext = null; try { // setup current namespace localContext = new VowpalWabbitJsonParseContext { Namespace = ns, Context = new VowpalWabbitMarshalContext(this.vw, context.ExampleBuilder), JsonProperty = ns.Name }; { this.defaultMarshaller.MarshalNamespace( localContext.Context, ns, () => this.ParseProperties(new List { localContext })); // append string features if we found some if (this.vw.Settings.EnableStringExampleGeneration) { context.StringExample .Append(localContext.Context.StringExample) .Append(string.Join(" ", this.namespaceStrings)); } } } finally { if (localContext != null && localContext.Context != null) { localContext.Context.Dispose(); localContext.Context = null; } } } } finally { this.DefaultNamespaceContext = saveDefaultNamespaceContext; } } /// /// Parses the example. /// /// The example to parse. /// /// Optional label, taking precedence over "_label" property found in . /// If null, will be inspected and the "_label" property used as label. /// /// Action to be executed when special properties are discovered. /// The VowpalWabbit native example. public void Parse(JsonReader reader, ILabel label = null, List extensions = null) { this.featureCount = 0; this.labelObject = null; this.foundMulti = false; // avoid parameter passing for the sake of non-reentrantness this.reader = reader; this.label = label; this.extensions = extensions; if (label != null) this.defaultMarshaller.MarshalLabel(this.DefaultNamespaceContext, label); // handle the case when the reader is already positioned at JsonToken.StartObject if (reader.TokenType == JsonToken.None && !reader.Read()) return; if (reader.TokenType != JsonToken.StartObject) throw new VowpalWabbitJsonException(this.reader, string.Format("Expected start object. Found '{0}' and value '{1}'", reader.TokenType, reader.Value)); var ns = new Namespace(this.vw); var path = new List { new VowpalWabbitJsonParseContext { Namespace = ns, Context = this.DefaultNamespaceContext, JsonProperty = string.Empty } }; this.extensionState.Reader = reader; this.extensionState.Path = path; // TODO: duplicate namespace recursion to enable async // featureCount might be modified inside ParseProperties... this.featureCount = this.defaultMarshaller.MarshalNamespace(this.DefaultNamespaceContext, ns, () => this.ParseProperties(path)) + this.featureCount; if (this.labelObject != null) { var propertyName = ((JProperty)this.labelObject.First).Name; Type labelType; if (!labelPropertyMapping.TryGetValue(propertyName.ToLowerInvariant(), out labelType)) throw new VowpalWabbitJsonException(this.reader, "The first property ('" + propertyName + "') must match to a property of a VowpalWabbit label type."); var labelObj = (ILabel)this.labelObject.ToObject(labelType); if (this.foundMulti) this.Label = labelObj; else this.defaultMarshaller.MarshalLabel(this.DefaultNamespaceContext, labelObj); } } private void ParseSpecialProperty(VowpalWabbitJsonParseContext context, string propertyName) { var propertyConfiguration = this.vw.Settings.PropertyConfiguration; // special fields if (propertyName.Equals(propertyConfiguration.LabelProperty, StringComparison.OrdinalIgnoreCase)) { // passed in label has precedence if (label == null) this.ParseLabel(); else reader.Skip(); } else if (propertyName.Equals(propertyConfiguration.TextProperty, StringComparison.OrdinalIgnoreCase)) { // parse text segment feature this.defaultMarshaller.MarshalFeatureStringSplit( context.Context, context.Namespace, new Feature(propertyName), reader.ReadAsString()); } else if (propertyName.Equals(propertyConfiguration.LabelIndexProperty, StringComparison.OrdinalIgnoreCase)) { if (!this.reader.Read()) throw new VowpalWabbitJsonException(this.reader, "Unexpected end"); // skip if (this.reader.TokenType == JsonToken.Null) return; this.LabelIndex = (int)(long)this.reader.Value; } else if (propertyName.StartsWith(propertyConfiguration.LabelPropertyPrefix, StringComparison.OrdinalIgnoreCase)) { if (!this.reader.Read()) throw new VowpalWabbitJsonException(this.reader, "Unexpected end"); // skip if (this.reader.TokenType == JsonToken.Null) return; if (this.labelObject == null) this.labelObject = new JObject(); var targetPropertyName = propertyName.Substring(propertyConfiguration.LabelPropertyPrefix.Length); this.labelObject.Add(targetPropertyName, new JValue(this.reader.Value)); } else { if (propertyName.Equals(propertyConfiguration.MultiProperty, StringComparison.Ordinal)) this.foundMulti = true; // forward to handler if (this.extensions != null) foreach (var extension in this.extensions) if (extension(this.extensionState, propertyName)) return; // if not handled, skip it reader.Skip(); } } private void ParseLabel() { // peak the first property name if (!this.reader.Read()) throw new VowpalWabbitJsonException(this.reader, "Unexpected end"); switch (reader.TokenType) { case JsonToken.StartObject: { // parse complex object if (!reader.Read() || reader.TokenType != JsonToken.PropertyName) throw new VowpalWabbitJsonException(this.reader, "Expected at least a single property to determine the label object"); var propertyName = (string)reader.Value; var prefixReader = new PrefixedJsonReader(this.reader, Tuple.Create(JsonToken.StartObject, (object)null), Tuple.Create(JsonToken.PropertyName, (object)propertyName)); Type labelType; if (!labelPropertyMapping.TryGetValue(propertyName.ToLowerInvariant(), out labelType)) throw new VowpalWabbitJsonException(this.reader, "The first property ('" + propertyName + "') must match to a property of a VowpalWabbit label type."); var label = (ILabel)jsonSerializer.Deserialize(prefixReader, labelType); this.defaultMarshaller.MarshalLabel(this.DefaultNamespaceContext, label); } break; case JsonToken.Integer: case JsonToken.Float: case JsonToken.String: { // pass label directly to VW var labelString = reader.Value.ToString(); this.defaultMarshaller.MarshalLabel(this.DefaultNamespaceContext, new StringLabel(labelString)); } break; case JsonToken.Null: // ignore break; default: throw new VowpalWabbitJsonException(this.reader, "Expected label object"); } } /// /// Expects that actual feature value. /// private void ParseFeature(List path, string featureName) { switch (featureName) { case "$id": { if (this.referenceResolver == null) return; var id = (string)reader.Value; if (!reader.Read() || reader.TokenType != JsonToken.PropertyName || (string)reader.Value != "$values") throw new VowpalWabbitJsonException(this.reader, "Expecting '$values' property"); // read $values if (!reader.Read()) throw new VowpalWabbitJsonException(this.reader, "Unexpected end"); // create re-useable marshalling call var marshalAction = this.ParseFeatureReUsable(); // keep action for re-use this.referenceResolver.AddReference(id, marshalAction); // go up 2 levels to find actual namespace, the last one is actually the property we want to serialize featureName = path.Last().JsonProperty; var context = path[path.Count - 2]; marshalAction.Marshal(this.defaultMarshaller, context.Context, context.Namespace, featureName); } return; case "$ref": { if (this.referenceResolver == null || this.serializer == null) return; var id = (string)reader.Value; // go up 2 levels to find actual namespace, the last one is actually the property we want to serialize featureName = path.Last().JsonProperty; var ns = path[path.Count - 2].Namespace; this.referenceResolver.Resolve( this.serializer, id, marshalAction => { // setup fresh context using (var context = new VowpalWabbitMarshalContext(this.vw, this.DefaultNamespaceContext.ExampleBuilder)) { this.featureCount += this.defaultMarshaller.MarshalNamespace( context, ns, () => marshalAction.Marshal(this.defaultMarshaller, context, ns, featureName)); // append default namespaces features if we found some if (this.vw.Settings.EnableStringExampleGeneration) { var str = context.ToString(); if (str.Length > 0) this.namespaceStrings.Add(str); } } }); } return; } var localContext = path.Last(); this.ParseFeature(path, localContext.Context, localContext.Namespace, featureName); } private IVowpalWabbitMarshalAction ParseFeatureReUsable() { // make sure the returned action is independent of the current parsing context, so we can ship it switch (reader.TokenType) { case JsonToken.Float: return VowpalWabbitMarshalActions.Create((double)reader.Value); case JsonToken.Integer: return VowpalWabbitMarshalActions.Create((long)reader.Value); case JsonToken.String: return VowpalWabbitMarshalActions.Create((string)reader.Value); case JsonToken.Boolean: return VowpalWabbitMarshalActions.Create((bool)reader.Value); case JsonToken.Comment: case JsonToken.Null: // probably best to ignore? break; case JsonToken.StartArray: return this.ParseFeatureArrayReUsable(); } return null; } /// /// Expects: "1,2.2,3]" (excluding the leading [) /// private IVowpalWabbitMarshalAction ParseFeatureArrayReUsable() { var values = new float[16]; var index = 0; while (reader.Read()) { float val; switch (reader.TokenType) { case JsonToken.Integer: val = (float)(long)reader.Value; break; case JsonToken.Float: val = (float)(double)reader.Value; break; case JsonToken.EndArray: goto done; default: throw new VowpalWabbitJsonException(this.reader, "Unxpected token " + reader.TokenType + " while deserializing dense feature array"); } if (index == values.Length) { var newValues = new float[values.Length * 2]; Array.Copy(values, newValues, values.Length); values = newValues; } values[index++] = val; } done: return VowpalWabbitMarshalActions.Create(values, index); } /// /// Expects that actual feature value. /// private void ParseFeature(List path, VowpalWabbitMarshalContext context, Namespace ns, string featureName) { switch (reader.TokenType) { case JsonToken.Float: VowpalWabbitMarshalActions.Marshal(this.defaultMarshaller, context, ns, featureName, (double)reader.Value); break; case JsonToken.Integer: VowpalWabbitMarshalActions.Marshal(this.defaultMarshaller, context, ns, featureName, (long)reader.Value); break; case JsonToken.String: VowpalWabbitMarshalActions.Marshal(this.defaultMarshaller, context, ns, featureName, (string)reader.Value); break; case JsonToken.Boolean: VowpalWabbitMarshalActions.Marshal(this.defaultMarshaller, context, ns, featureName, (bool)reader.Value); break; case JsonToken.Comment: case JsonToken.Null: // probably best to ignore? break; case JsonToken.StartArray: this.WrapInNamespace(path, featureName, lastContext => this.ParseFeatureArray(path)); break; default: throw new VowpalWabbitJsonException(this.reader, "Unexpected token " + reader.TokenType + " while deserializing primitive feature"); } } /// /// Expects: "1,2.2,3]" (excluding the leading [) /// private void ParseFeatureArray(List path) { var context = path.Last().Context; var ns = path.Last().Namespace; ulong index = 0; while (reader.Read()) { switch (reader.TokenType) { case JsonToken.Integer: MarshalFloatFeature(context, ns, index, (float)(long)reader.Value); break; case JsonToken.Float: MarshalFloatFeature(context, ns, index, (float)(double)reader.Value); break; case JsonToken.StartObject: ParseProperties(path); break; case JsonToken.EndArray: return; case JsonToken.Null: // just ignore nulls break; default: throw new VowpalWabbitJsonException(this.reader, "Unxpected token " + reader.TokenType + " while deserializing dense feature array"); } index++; } } private static void MarshalFloatFeature(VowpalWabbitMarshalContext context, Namespace ns, ulong index, float value) { context.NamespaceBuilder.AddFeature(ns.NamespaceHash + index, value); if (context.StringExample != null) { context.AppendStringExample( false, " {0}:" + (context.VW.Settings.EnableStringFloatCompact ? "{1}" : "{1:E20}"), index, value); } } private void WrapInNamespace(List path, string namespaceValue, Action action) { VowpalWabbitJsonParseContext parseContext = null; VowpalWabbitMarshalContext marshalContext = null; try { var ns = new Namespace(this.vw, namespaceValue); marshalContext = new VowpalWabbitMarshalContext(this.vw, this.DefaultNamespaceContext.ExampleBuilder); parseContext = new VowpalWabbitJsonParseContext { Namespace = ns, Context = marshalContext, JsonProperty = namespaceValue }; // the namespace is only added on dispose, to be able to check if at least a single feature has been added marshalContext.NamespaceBuilder = marshalContext.ExampleBuilder.AddNamespace(ns.FeatureGroup); var position = 0; var stringExample = marshalContext.StringExample; if (marshalContext.StringExample != null) position = stringExample.Append(ns.NamespaceString).Length; path.Add(parseContext); action(parseContext); // append default namespaces features if we found some if (this.vw.Settings.EnableStringExampleGeneration) { var str = marshalContext.ToString(); if (str.Length > 0) this.namespaceStrings.Add(str); } this.featureCount += (int)marshalContext.NamespaceBuilder.FeatureCount; } finally { path.RemoveAt(path.Count - 1); if (marshalContext.NamespaceBuilder != null) { marshalContext.NamespaceBuilder.Dispose(); marshalContext.NamespaceBuilder = null; } if (parseContext != null && parseContext.Context != null) { parseContext.Context.Dispose(); parseContext.Context = null; } } } /// /// Parses { "feature1":1, "feature2":true, .... } /// private void ParseNamespaceAndFeatures(List path, string namespaceValue) { this.WrapInNamespace(path, namespaceValue, context => this.ParseProperties(path)); } private void ParseProperties(List path) { var propertyConfiguration = this.vw.Settings.PropertyConfiguration; while (reader.Read()) { switch (reader.TokenType) { case JsonToken.PropertyName: var propertyName = (string)reader.Value; if (propertyName.StartsWith(propertyConfiguration.FeatureIgnorePrefix, StringComparison.Ordinal) || propertyConfiguration.IsSpecialProperty(propertyName)) { this.ParseSpecialProperty(path.Last(), propertyName); continue; } if (!reader.Read()) throw new VowpalWabbitJsonException(this.reader, "Unexpected end while parsing namespace"); // TODO: this.Context might have to be a stack... if (reader.TokenType == JsonToken.StartObject) this.ParseNamespaceAndFeatures(path, propertyName); else this.ParseFeature(path, propertyName); break; case JsonToken.EndObject: return; } } } /// /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources. /// public void Dispose() { this.Dispose(true); GC.SuppressFinalize(this); } private void Dispose(bool disposing) { if (disposing) { if (this.DefaultNamespaceContext != null) { // useful for tracking down bugs // this.DefaultNamespaceContextStackTrace = "Dispose" + Environment.StackTrace; this.DefaultNamespaceContext.Dispose(); this.DefaultNamespaceContext = null; } } } } /// /// A parsing context holding the current state during JSON parsing. /// public sealed class VowpalWabbitJsonParseContext { /// /// The current marshalling context. /// public VowpalWabbitMarshalContext Context { get; set; } /// /// The current namespace. /// public Namespace Namespace { get; set; } /// /// The current JSON property being processed. /// public string JsonProperty { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs_json/Serializer/VowpalWabbitJsonException.cs000066400000000000000000000037061323713302100271230ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Newtonsoft.Json; using System; namespace VW.Serializer { /// /// Exception thrown if fails to deserialize the JSON. /// [Serializable] public sealed class VowpalWabbitJsonException : Exception { /// /// Initializes a new instance of the class. /// /// The reader used at deserialization time. /// The message that describes the error. public VowpalWabbitJsonException(JsonReader reader, string message) : base(message) { this.Path = reader.Path; var lineInfo = reader as IJsonLineInfo; if (lineInfo != null) { this.LineNumber = lineInfo.LineNumber; this.LinePosition = lineInfo.LinePosition; } } /// /// The line number at which this error happened. /// public int LineNumber { get; private set; } /// /// The character position at which this error happened. /// public int LinePosition { get; private set; } /// /// The path as returned by . /// public string Path { get; private set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs_json/Serializer/VowpalWabbitJsonOptimizedSerializable.cs000066400000000000000000000047451323713302100314640ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Newtonsoft.Json; using System; using System.IO; using VW.Serializer.Intermediate; namespace VW.Serializer { /// /// Uses the supplied to get the JSON fragments for a given object. /// public class VowpalWabbitJsonOptimizedSerializable : IVowpalWabbitSerializable { private readonly object value; private readonly IVowpalWabbitJsonConverter jsonConverter; /// /// Initializes a new instance. /// public VowpalWabbitJsonOptimizedSerializable(object value, IVowpalWabbitJsonConverter jsonConverter) { this.value = value; this.jsonConverter = jsonConverter; } /// /// Marshals JSON string into VW example. /// public void Marshal(VowpalWabbitMarshalContext ctx, Namespace ns, Feature feature) { if (this.value == null) return; try { var jsonSerializer = new JsonSerializer(); using (var jsonBuilder = new VowpalWabbitJsonBuilder(ctx.VW, VowpalWabbitDefaultMarshaller.Instance, jsonSerializer)) { // marshal from JSON to VW foreach (var json in jsonConverter.JsonFragments(this.value)) { if (json == null) continue; using (var reader = new JsonTextReader(new StringReader(json))) { jsonBuilder.Parse(reader, ctx, new Namespace(ctx.VW, feature.Name)); } } } } catch (Exception e) { throw new VowpalWabbitSerializationException("Optimized marshalling failed", e, ns, feature); } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs_json/Serializer/VowpalWabbitJsonReferenceResolver.cs000066400000000000000000000206341323713302100306040ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Linq; using System.Runtime.Caching; namespace VW.Serializer { /// /// Reference resolver for JSON.NET $id, $ref elements. /// public sealed class VowpalWabbitJsonReferenceResolver : IDisposable { /// /// Monitoring statistics. /// public sealed class Stats { /// /// The number of items currently cached. /// public long ItemCount { get; internal set; } /// /// The number of outstanding requests to resolve a referencce. /// public long NumberOfOpenRequests { get; internal set; } } private readonly Action exampleComplete; private readonly object lockObject; private MemoryCache cache; private MemoryCache cacheRequests; private readonly Func cacheItemPolicyFactory; private readonly Func cacheRequestItemPolicyFactory; private int numberOfOpenRequests; /// /// Initializes a new instance. /// /// A callback triggered when all outstanding references for a given example are resolved. /// Optional name. /// Optional cache policy for cached items. Defaults to 1 hour sliding expiration. /// Optional cache policy for resolution requets. Defaults to 1 hour sliding expiration. public VowpalWabbitJsonReferenceResolver( Action exampleComplete, string cacheName = null, Func cacheItemPolicyFactory = null, Func cacheRequestItemPolicyFactory = null) { this.lockObject = new object(); this.exampleComplete = exampleComplete; if (cacheName == null) cacheName = "VowpalWabbitJsonExampleCache"; this.cacheItemPolicyFactory = cacheItemPolicyFactory == null ? _ => new CacheItemPolicy { SlidingExpiration = TimeSpan.FromHours(1) } : cacheItemPolicyFactory; this.cacheRequestItemPolicyFactory = cacheRequestItemPolicyFactory == null ? _ => new CacheItemPolicy { SlidingExpiration = TimeSpan.FromHours(1) } : cacheRequestItemPolicyFactory; this.cache = new MemoryCache(cacheName); this.cacheRequests = new MemoryCache(cacheName + "Requests"); } /// /// Monitoring statistics. /// public Stats Statistics { get { lock (this.lockObject) { return new Stats { ItemCount = this.cache.GetCount(), NumberOfOpenRequests = this.numberOfOpenRequests }; } } } internal void AddReference(string id, IVowpalWabbitMarshalAction marshalAction) { List requests = null; lock (this.lockObject) { // ignore duplicate keys - still update the sliding timer if (this.cache.Contains(id)) return; this.cache.Add( new CacheItem(id, marshalAction), this.cacheItemPolicyFactory(id)); requests = (List)this.cacheRequests.Get(id); if (requests != null) { foreach (var req in requests) req.DontDispose = true; this.cacheRequests.Remove(id); this.numberOfOpenRequests -= requests.Count; } } // since this can be called from another thread we need to dispatch to the serializer and let it decide // when to resolve the marshalling request if (requests != null) { foreach (var req in requests) if (req.Serializer.Resolve(() => req.Marshal(marshalAction))) this.exampleComplete(req.Serializer); } } internal void Resolve(VowpalWabbitJsonSerializer serializer, string id, Action resolveAction) { IVowpalWabbitMarshalAction marshal; lock (this.lockObject) { marshal = (IVowpalWabbitMarshalAction)this.cache.Get(id); if (marshal == null) { // not found, register for delayed completion var requests = (List)this.cacheRequests.Get(id); if (requests == null) { var policy = this.cacheRequestItemPolicyFactory(id); // dispatch to original handler too var removeHandler = policy.RemovedCallback; if (removeHandler == null) policy.RemovedCallback = this.CacheEntryRemovedCallback; else policy.RemovedCallback = args => { removeHandler(args); this.CacheEntryRemovedCallback(args); }; requests = new List(); this.cacheRequests.Add( new CacheItem(id, requests), policy); } requests.Add( new IncompleteReferenceRequest { Serializer = serializer, Marshal = resolveAction }); this.numberOfOpenRequests++; serializer.IncreaseUnresolved(); return; } } // avoid extensive locking resolveAction(marshal); } private void CacheEntryRemovedCallback(CacheEntryRemovedArguments arguments) { lock (this.lockObject) { var requests = (List)arguments.CacheItem.Value; // dispose outstanding requests foreach (var request in requests) if (!request.DontDispose) request.Serializer.Dispose(); } } private sealed class IncompleteReferenceRequest { internal IncompleteReferenceRequest() { this.DontDispose = false; } internal VowpalWabbitJsonSerializer Serializer { get; set; } internal Action Marshal { get; set; } // if we return to the handler, the handler has to dispose internal bool DontDispose { get; set; } } /// /// Disposes hold resources. /// public void Dispose() { if (this.cacheRequests != null) { // trigger dispose foreach (var key in this.cacheRequests.Select(kv => kv.Key).ToList()) this.cacheRequests.Remove(key); this.cacheRequests.Dispose(); this.cacheRequests = null; } if (this.cache != null) { this.cache.Dispose(); this.cache = null; } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs_json/Serializer/VowpalWabbitJsonSerializable.cs000066400000000000000000000045321323713302100275710ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Newtonsoft.Json; using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Text; using System.Threading.Tasks; using VW.Serializer.Intermediate; namespace VW.Serializer { /// /// Uses supplied JsonConverter to custom serialize object into JSON and then marshal from there. /// public class VowpalWabbitJsonSerializable : IVowpalWabbitSerializable { private readonly object value; private readonly JsonConverter jsonConverter; /// /// Initializes a new instance. /// public VowpalWabbitJsonSerializable(object value, JsonConverter jsonConverter) { this.value = value; this.jsonConverter = jsonConverter; } /// /// Marshals JSON string into VW example. /// public void Marshal(VowpalWabbitMarshalContext ctx, Namespace ns, Feature feature) { if (this.value == null) return; var jsonSerializer = new JsonSerializer(); using (var jsonBuilder = new VowpalWabbitJsonBuilder(ctx.VW, VowpalWabbitDefaultMarshaller.Instance, jsonSerializer)) { // serialize from object to JSON var sb = new StringBuilder(); using (var writer = new JsonTextWriter(new StringWriter(sb))) { this.jsonConverter.WriteJson(writer, this.value, jsonSerializer); } // marshal from JSON to VW using (var reader = new JsonTextReader(new StringReader(sb.ToString()))) { jsonBuilder.Parse(reader, ctx, new Namespace(ctx.VW, feature.Name)); } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs_json/Serializer/VowpalWabbitJsonSerializer.cs000066400000000000000000000470001323713302100272710ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Newtonsoft.Json; using System; using System.Collections.Generic; using System.Diagnostics.Contracts; using System.Globalization; using System.IO; using System.Linq; using System.Threading.Tasks; using VW.Labels; using VW.Serializer.Intermediate; namespace VW.Serializer { /// /// The current JSON parse state. /// public sealed class VowpalWabbitJsonParseState { /// /// The native VW instance. /// public VowpalWabbit VW { get; set; } /// /// The JSON reader. /// public JsonReader Reader { get; set; } /// /// The VW example JSON builder. /// public VowpalWabbitJsonBuilder JsonBuilder { get; set; } /// /// The current property path within the JSON. /// public List Path { get; set; } /// /// The current _multi element index. /// public int MultiIndex { get; set; } /// /// Triggers parsing at the current state of the using the default namespace. /// public void Parse() { using (var context = new VowpalWabbitMarshalContext(this.VW, this.JsonBuilder.DefaultNamespaceContext.ExampleBuilder)) { var ns = new Namespace(this.VW); this.Parse(context, ns); } } /// /// Triggers parsing at the current state of the using the given . /// /// The namespace the JSON should be marshalled into. /// The namespace the JSON should be marshalled into. public void Parse(VowpalWabbitMarshalContext namespaceContext, Namespace ns) { this.JsonBuilder.Parse(this.Path, namespaceContext, ns); } } /// /// Delegate definition for JSON parsing extension. E.g. if one wants to extract "_timestamp" or a like. /// /// The current parsing state. /// The property encountered. /// True if the extension handled this property, false otherwise. /// Only fires for "ignore prefixed" properties. public delegate bool VowpalWabbitJsonExtension(VowpalWabbitJsonParseState state, string property); /// /// A deserializer from JSON to Vowpal Wabbit native examples. /// public sealed class VowpalWabbitJsonSerializer : IDisposable { private readonly IVowpalWabbitExamplePool vwPool; private readonly JsonSerializer jsonSerializer; private readonly VowpalWabbitJsonReferenceResolver referenceResolver; private int unresolved; private readonly object lockObject = new object(); private bool ready = false; private List marshalRequests; private List extensions; /// /// Initializes a new instance of the class. /// /// The VW native instance. /// An optional reference resolver. public VowpalWabbitJsonSerializer(IVowpalWabbitExamplePool vwPool, VowpalWabbitJsonReferenceResolver referenceResolver = null) { Contract.Requires(vwPool != null); this.extensions = new List { this.HandleMultiProperty }; this.jsonSerializer = new JsonSerializer(); this.vwPool = vwPool; this.referenceResolver = referenceResolver; this.ExampleBuilder = new VowpalWabbitJsonBuilder(this, this.vwPool, VowpalWabbitDefaultMarshaller.Instance, this.jsonSerializer); } /// /// Registers a parsing extension. /// /// The extension to be rgistered. public void RegisterExtension(VowpalWabbitJsonExtension extension) { this.extensions.Add(extension); } /// /// Userful if this deserializer is published through VowpalWabbitJsonReferenceResolver. /// public object UserContext { get; set; } /// /// Single line example or shared example. /// public VowpalWabbitJsonBuilder ExampleBuilder { get; private set; } /// /// Multi-line examples. /// public List ExampleBuilders { get; private set; } internal VowpalWabbitJsonReferenceResolver ReferenceResolver { get { return this.referenceResolver; } } internal void IncreaseUnresolved() { // only called during the initial parsing run this.unresolved++; } internal bool Resolve(Action marshal) { lock (this.lockObject) { // ready is false until the initial parsing run is complete if (this.ready) { // the object doesn't get anymore unresolved marshal requests if (this.marshalRequests != null) { foreach (var req in this.marshalRequests) req(); this.unresolved -= this.marshalRequests.Count; this.marshalRequests = null; } marshal(); this.unresolved--; if (this.unresolved < 0) throw new InvalidOperationException("Number of unresolved requested must not be negative"); return this.unresolved == 0; } else { // we need to track the requests and wait until the initial parsing is done if (this.marshalRequests == null) this.marshalRequests = new List(); this.marshalRequests.Add(marshal); return false; } } } /// /// Creates the VW example, be it single or multi-line. /// /// The marshalled VW example. public VowpalWabbitExampleCollection CreateExamples() { lock (this.lockObject) { if (this.unresolved == 0) return this.CreateExamplesInternal(); if (this.marshalRequests != null && this.unresolved == this.marshalRequests.Count) { return this.CreateExamplesInternal(); } // wait for delayed completion this.ready = true; return null; } } /// /// Creates the VW example, be it single or multi-line. /// /// The label to be applied. /// The index of the example in the multi-line example this label should be applied on. /// public VowpalWabbitExampleCollection CreateExamples(ILabel label, int index) { if (index >= this.ExampleBuilders.Count) throw new InvalidDataException($"Label index {index} is invalid. Only {this.ExampleBuilders.Count} examples available."); VowpalWabbitDefaultMarshaller.Instance.MarshalLabel( this.ExampleBuilders[index].DefaultNamespaceContext, label); return this.CreateExamples(); } /// /// Parses and creates the example. /// /// The example to parse. /// /// Optional label, taking precedence over "_label" property found in . /// If null, will be inspected and the "_label" property used as label. /// /// Optional index of example the given label should be applied for multi-line examples. /// The VowpalWabbit native example. public VowpalWabbitExampleCollection ParseAndCreate(string json, ILabel label = null, int? index = null) { this.Parse(json, label, index); return this.CreateExamples(); } /// /// Parses the example. /// /// The example to parse. /// /// Optional label, taking precedence over "_label" property found in . /// If null, will be inspected and the "_label" property used as label. /// /// Optional index of example the given label should be applied for multi-line examples. /// The VowpalWabbit native example. public VowpalWabbitExampleCollection ParseAndCreate(JsonReader reader, ILabel label = null, int? index = null) { this.Parse(reader, label, index); return this.CreateExamples(); } /// /// Parses the example. /// /// The example to parse. /// /// Optional label, taking precedence over "_label" property found in . /// If null, will be inspected and the "_label" property used as label. /// /// Optional index of example the given label should be applied for multi-line examples. public void Parse(string json, ILabel label = null, int? index = null) { using (var textReader = new JsonTextReader(new StringReader(json))) { this.Parse(textReader, label); } } /// /// Returns the number of action dependent examples found within . /// /// The JSON to be inspected. /// Returns the number of action dependent examples. public static int GetNumberOfActionDependentExamples(string json) { using (var textReader = new JsonTextReader(new StringReader(json))) { return GetNumberOfActionDependentExamples(textReader); } } /// /// Returns the number of action dependent examples found within . /// /// The JSON. /// The optional multi property name. /// Returns the number of action dependent examples. public static int GetNumberOfActionDependentExamples(JsonReader reader, string multiProperty = PropertyConfiguration.MultiPropertyDefault) { // handle the case when the reader is already positioned at JsonToken.StartObject if (reader.TokenType == JsonToken.None && !reader.Read()) throw new VowpalWabbitJsonException(reader, "Expected non-empty JSON"); if (reader.TokenType != JsonToken.StartObject) throw new VowpalWabbitJsonException(reader, "Expected start object"); while (reader.Read()) { if (!(reader.TokenType == JsonToken.PropertyName && (string)reader.Value == multiProperty)) { reader.Skip(); continue; } if (!reader.Read() || reader.TokenType != JsonToken.StartArray) throw new VowpalWabbitJsonException(reader, "Expected start arrray"); var exampleCount = 0; while (reader.Read() && reader.TokenType != JsonToken.EndArray) { exampleCount++; reader.Skip(); } return exampleCount; } return 0; } private bool HandleMultiProperty(VowpalWabbitJsonParseState state, string property) { var multiPropertyName = this.vwPool.Native.Settings.PropertyConfiguration.MultiProperty; if (!property.Equals(multiPropertyName, StringComparison.OrdinalIgnoreCase)) return false; var reader = state.Reader; if (!reader.Read() || reader.TokenType != JsonToken.StartArray) throw new VowpalWabbitJsonException(reader, "Expected start array for '" + multiPropertyName + "'"); if (this.ExampleBuilders == null) this.ExampleBuilders = new List(); state.MultiIndex = 0; while (reader.Read()) { switch (reader.TokenType) { case JsonToken.StartObject: VowpalWabbitJsonBuilder builder = null; try { builder = new VowpalWabbitJsonBuilder(this, this.vwPool, VowpalWabbitDefaultMarshaller.Instance, this.jsonSerializer, state.MultiIndex); this.ExampleBuilders.Add(builder); } catch (Exception) { builder.Dispose(); throw; } // pass the label to the selected example builder.Parse(reader, index != null && index == this.ExampleBuilders.Count - 1 ? label : null, this.extensions); state.MultiIndex++; break; case JsonToken.EndArray: return true; default: throw new VowpalWabbitJsonException(reader, "Unexpected token: " + reader.TokenType); } } throw new VowpalWabbitJsonException(reader, "Unexpected end"); } // TODO: keeping it local might be nicer... private int? index; private ILabel label; /// /// Parses the example. /// /// The example to parse. /// /// Optional label, taking precedence over "_label" property found in . /// If null, will be inspected and the "_label" property used as label. /// /// Optional index of example the given label should be applied for multi-line examples. public void Parse(JsonReader reader, ILabel label = null, int? index = null) { this.index = index; this.label = label; // only pass the label if it's not targeted at a particular index this.ExampleBuilder.Parse(reader, index == null ? label : null, this.extensions); // check if the outer example found a label if (this.ExampleBuilder.Label != null) { if (this.ExampleBuilder.LabelIndex >= this.ExampleBuilders.Count) throw new InvalidDataException($"Label index {this.ExampleBuilder.LabelIndex} is invalid. Only {this.ExampleBuilders.Count} examples available."); VowpalWabbitDefaultMarshaller.Instance.MarshalLabel( this.ExampleBuilders[this.ExampleBuilder.LabelIndex].DefaultNamespaceContext, this.ExampleBuilder.Label); } } /// /// Creates the examples ready for learning or prediction. /// public VowpalWabbitExampleCollection CreateExamplesInternal() { try { if (this.ExampleBuilders == null) { return new VowpalWabbitSingleLineExampleCollection(this.vwPool.Native, this.ExampleBuilder.CreateExample()); } else { // making sure we don't leak memory VowpalWabbitExample sharedExample = null; var examples = new VowpalWabbitExample[this.ExampleBuilders.Count]; try { // mark shared example as shared VowpalWabbitDefaultMarshaller.Instance.MarshalLabel(this.ExampleBuilder.DefaultNamespaceContext, SharedLabel.Instance); sharedExample = this.ExampleBuilder.CreateExample(); for (int i = 0; i < this.ExampleBuilders.Count; i++) examples[i] = this.ExampleBuilders[i].CreateExample(); return new VowpalWabbitMultiLineExampleCollection(this.vwPool.Native, sharedExample, examples); } catch (Exception) { if (sharedExample != null) sharedExample.Dispose(); foreach (var e in examples) if (e != null) e.Dispose(); throw; } } } finally { this.ExampleBuilder.Dispose(); this.ExampleBuilder = null; if (this.ExampleBuilders != null) { foreach (var eb in this.ExampleBuilders) eb.Dispose(); this.ExampleBuilders = null; } } } /// /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources. /// public void Dispose() { this.Dispose(true); GC.SuppressFinalize(this); } private void Dispose(bool disposing) { // Remark: might be called multiple times from VowpalWabbitJsonReferenceResolver if (disposing) { // cleanup in case CreateExample() wasn't called if (this.ExampleBuilder != null) { this.ExampleBuilder.Dispose(); this.ExampleBuilder = null; } if (this.ExampleBuilders != null) { foreach (var eb in this.ExampleBuilders) eb.Dispose(); this.ExampleBuilders = null; } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs_json/VowpalWabbitJson.cs000066400000000000000000000261521323713302100231330ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Newtonsoft.Json; using System; using System.Diagnostics.Contracts; using VW.Labels; using VW.Serializer; namespace VW { /// /// A VowpalWabbit wrapper reading from JSON (see https://github.com/JohnLangford/vowpal_wabbit/wiki/JSON) /// public sealed class VowpalWabbitJson : IDisposable { private VowpalWabbit vw; /// /// Initializes a new instance of the class. /// /// Command line arguments passed to native instance. public VowpalWabbitJson(String args) : this(new VowpalWabbit(args)) { } /// /// Initializes a new instance of the class. /// /// Arguments passed to native instance. public VowpalWabbitJson(VowpalWabbitSettings settings) : this(new VowpalWabbit(settings)) { } /// /// Initializes a new instance of the class. /// /// The native instance to wrap. /// This instance takes ownership of instance and disposes it. public VowpalWabbitJson(VowpalWabbit vw) { if (vw == null) { throw new ArgumentNullException("vw"); } Contract.EndContractBlock(); this.vw = vw; } /// /// The wrapped VW instance. /// public VowpalWabbit Native { get { return this.vw; } } /// /// Learns from the given example. /// /// The example to learn. /// /// Optional label, taking precedence over "_label" property found in . /// If null, will be inspected and the "_label" property used as label. /// /// Optional index of example the given label should be applied for multi-line examples. public void Learn(string json, ILabel label = null, int? index = null) { using (var serializer = new VowpalWabbitJsonSerializer(vw)) using (var result = serializer.ParseAndCreate(json, label, index)) { result.Learn(); } } /// /// Learns from the given example. /// /// The example to learn. /// /// Optional label, taking precedence over "_label" property found in . /// If null, will be inspected and the "_label" property used as label. /// /// Optional index of example the given label should be applied for multi-line examples. public void Learn(JsonReader reader, ILabel label = null, int? index = null) { using (var serializer = new VowpalWabbitJsonSerializer(vw)) using (var result = serializer.ParseAndCreate(reader, label, index)) { result.Learn(); } } /// /// Learn from the given example and return the current prediction for it. /// /// The prediction type. /// The example to learn. /// The prediction factory to be used. See . /// /// Optional label, taking precedence over "_label" property found in . /// If null, will be inspected and the "_label" property used as label. /// /// Optional index of example the given label should be applied for multi-line examples. /// The prediction for the given . public TPrediction Learn(string json, IVowpalWabbitPredictionFactory predictionFactory, ILabel label = null, int? index = null) { using (var serializer = new VowpalWabbitJsonSerializer(vw)) using (var result = serializer.ParseAndCreate(json, label, index)) { return result.Learn(predictionFactory); } } /// /// Learn from the given example and return the current prediction for it. /// /// The prediction type. /// The example to learn. /// The prediction factory to be used. See . /// /// Optional label, taking precedence over "_label" property found in . /// If null, will be inspected and the "_label" property used as label. /// /// Optional index of example the given label should be applied for multi-line examples. /// The prediction for the given . public TPrediction Learn(JsonReader reader, IVowpalWabbitPredictionFactory predictionFactory, ILabel label = null, int? index = null) { using (var serializer = new VowpalWabbitJsonSerializer(vw)) using (var result = serializer.ParseAndCreate(reader, label, index)) { return result.Learn(predictionFactory); } } /// /// Predicts for the given example. /// /// The example to predict for. /// /// Optional label, taking precedence over "_label" property found in . /// If null, will be inspected and the "_label" property used as label. /// /// Optional index of example the given label should be applied for multi-line examples. public void Predict(string json, ILabel label = null, int? index = null) { Contract.Requires(json != null); using (var serializer = new VowpalWabbitJsonSerializer(vw)) using (var result = serializer.ParseAndCreate(json, label, index)) { result.Predict(); } } /// /// Predicts for the given example. /// /// The example to predict for. /// /// Optional label, taking precedence over "_label" property found in . /// If null, will be inspected and the "_label" property used as label. /// /// Optional index of example the given label should be applied for multi-line examples. public void Predict(JsonReader reader, ILabel label = null, int? index = null) { Contract.Requires(reader != null); using (var serializer = new VowpalWabbitJsonSerializer(vw)) using (var result = serializer.ParseAndCreate(reader, label, index)) { result.Predict(); } } /// /// Predicts for the given example. /// /// The prediction type. /// The example to predict for. /// The prediction factory to be used. See . /// /// Optional label, taking precedence over "_label" property found in . /// If null, will be inspected and the "_label" property used as label. /// /// Optional index of example the given label should be applied for multi-line examples. public TPrediction Predict(string json, IVowpalWabbitPredictionFactory predictionFactory, ILabel label = null, int? index = null) { Contract.Requires(json != null); using (var serializer = new VowpalWabbitJsonSerializer(vw)) using (var result = serializer.ParseAndCreate(json, label, index)) { return result.Predict(predictionFactory); } } /// /// Predicts for the given example. /// /// The prediction type. /// The example to predict for. /// The prediction factory to be used. See . /// /// Optional label, taking precedence over "_label" property found in . /// If null, will be inspected and the "_label" property used as label. /// /// Optional index of example the given label should be applied for multi-line examples. public TPrediction Predict(JsonReader reader, IVowpalWabbitPredictionFactory predictionFactory, ILabel label = null, int? index = null) { using (var serializer = new VowpalWabbitJsonSerializer(vw)) using (var result = serializer.ParseAndCreate(reader, label, index)) { return result.Predict(predictionFactory); } } /// /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources. /// public void Dispose() { this.Dispose(true); GC.SuppressFinalize(this); } private void Dispose(bool disposing) { if (disposing) { if (this.vw != null) { this.vw.Dispose(); this.vw = null; } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs_json/VowpalWabbitJsonThreadedPrediction.cs000066400000000000000000000027021323713302100266100ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- namespace VW { /// /// Enables multi-threaded prediction by utilizing a pool of instances. /// public sealed class VowpalWabbitJsonThreadedPrediction : VowpalWabbitThreadedPredictionBase { /// /// Initializes a new instance of . /// /// The model used by each pool instance. public VowpalWabbitJsonThreadedPrediction(VowpalWabbitModel model = null) : base(model) { } /// /// Creates a new instance of . /// /// The wrapped vw instance. protected override VowpalWabbitJson InternalCreate(VowpalWabbit vw) { return new VowpalWabbitJson(vw); } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs_json/VowpalWabbitMarshalAction.cs000066400000000000000000000121411323713302100247400ustar00rootroot00000000000000using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using VW.Serializer.Intermediate; namespace VW.Serializer { internal interface IVowpalWabbitMarshalAction { void Marshal(VowpalWabbitDefaultMarshaller defaultMarshaller, VowpalWabbitMarshalContext context, Namespace ns, string featureName); } internal static class VowpalWabbitMarshalActions { internal static IVowpalWabbitMarshalAction Create(double data) { return new VowpalWabbitMarshalActionImpl(Marshal, data); } internal static IVowpalWabbitMarshalAction Create(string data) { return new VowpalWabbitMarshalActionImpl(Marshal, data); } internal static IVowpalWabbitMarshalAction Create(long data) { return new VowpalWabbitMarshalActionImpl(Marshal, data); } internal static IVowpalWabbitMarshalAction Create(bool data) { return new VowpalWabbitMarshalActionImpl(Marshal, data); } internal static IVowpalWabbitMarshalAction Create(float[] data, int length) { return new VowpalWabbitMarshalActionArrayImpl(Marshal, data, length); } internal static void Marshal(VowpalWabbitDefaultMarshaller defaultMarshaller, VowpalWabbitMarshalContext context, Namespace ns, string featureName, double val) { var feature = new PreHashedFeature(context.VW, ns, featureName); defaultMarshaller.MarshalFeature(context, ns, feature, val); } internal static void Marshal(VowpalWabbitDefaultMarshaller defaultMarshaller, VowpalWabbitMarshalContext context, Namespace ns, string featureName, string val) { var feature = new Feature(featureName); defaultMarshaller.MarshalFeatureStringEscapeAndIncludeName(context, ns, feature, val); } internal static void Marshal(VowpalWabbitDefaultMarshaller defaultMarshaller, VowpalWabbitMarshalContext context, Namespace ns, string featureName, bool val) { var feature = new PreHashedFeature(context.VW, ns, featureName); defaultMarshaller.MarshalFeature(context, ns, feature, val); } internal static void Marshal(VowpalWabbitDefaultMarshaller defaultMarshaller, VowpalWabbitMarshalContext context, Namespace ns, string featureName, long val) { var feature = new PreHashedFeature(context.VW, ns, featureName); defaultMarshaller.MarshalFeature(context, ns, feature, val); } internal static void Marshal(VowpalWabbitDefaultMarshaller defaultMarshaller, VowpalWabbitMarshalContext context, Namespace ns, string featureName, float[] values, int length) { var feature = new Feature(featureName); defaultMarshaller.MarshalFeature(context, ns, feature, values, 0, length); } /// /// Explicit closure to enable debug view /// private sealed class VowpalWabbitMarshalActionImpl : IVowpalWabbitMarshalAction { internal delegate void MarshalAction(VowpalWabbitDefaultMarshaller defaultMarshaller, VowpalWabbitMarshalContext context, Namespace ns, string featureName, T data); private readonly T data; private readonly MarshalAction marshal; internal VowpalWabbitMarshalActionImpl(MarshalAction marshal, T data) { this.data = data; this.marshal = marshal; } public void Marshal(VowpalWabbitDefaultMarshaller defaultMarshaller, VowpalWabbitMarshalContext context, Namespace ns, string featureName) { this.marshal(defaultMarshaller, context, ns, featureName, this.data); } } /// /// Explicit closure to enable debug view /// private sealed class VowpalWabbitMarshalActionArrayImpl : IVowpalWabbitMarshalAction { internal delegate void MarshalAction(VowpalWabbitDefaultMarshaller defaultMarshaller, VowpalWabbitMarshalContext context, Namespace ns, string featureName, T[] data, int length); private readonly T[] data; private readonly int length; private readonly MarshalAction marshal; internal VowpalWabbitMarshalActionArrayImpl( MarshalAction marshal, T[] data, int length) { this.data = data; this.length = length; this.marshal = marshal; } public void Marshal(VowpalWabbitDefaultMarshaller defaultMarshaller, VowpalWabbitMarshalContext context, Namespace ns, string featureName) { this.marshal(defaultMarshaller, context, ns, featureName, this.data, this.length); } } } internal delegate void VowpalWabbitMarshalAction(VowpalWabbitDefaultMarshaller defaultMarshaller, VowpalWabbitMarshalContext context, Namespace ns, string featureName); } vowpal-wabbit-8.5.0.dfsg1/cs/cs_json/cs_json.csproj000066400000000000000000000150241323713302100222250ustar00rootroot00000000000000 Debug AnyCPU {9E27FA94-AB34-4736-8427-FB7A2BA90D52} Library Properties VW VowpalWabbit.JSON v4.5.2 512 ..\..\vowpalwabbit\ true ..\..\vowpalwabbit\x64\Debug\ DEBUG;TRACE full x64 prompt MinimumRecommendedRules.ruleset ..\..\vowpalwabbit\x64\Debug\VowpalWabbit.JSON.XML ..\..\vowpalwabbit\x64\Release\ TRACE true pdbonly x64 prompt MinimumRecommendedRules.ruleset ..\..\vowpalwabbit\x64\Release\VowpalWabbit.JSON.XML true ..\vw_key.snk $(SolutionDir)\packages\Newtonsoft.Json.9.0.1\lib\net45\Newtonsoft.Json.dll True {85e55ae0-3784-4968-9271-c81af560e1c1} vw_clr {e621e022-c1f8-433f-905a-ab9a3de072b7} vw_common {e4e962ae-7056-4eb0-a8c5-8dc824a4b068} cs This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. vowpal-wabbit-8.5.0.dfsg1/cs/cs_json/cs_json.nuspec000066400000000000000000000024211323713302100222170ustar00rootroot00000000000000 Vowpal Wabbit JSON VowpalWabbit.JSON vw vowpal wabbit langford ml machine learning John Langford et al $version$ https://github.com/JohnLangford/vowpal_wabbit/wiki/C%23-Binding https://github.com/JohnLangford/vowpal_wabbit/blob/master/LICENSE false Official Vowpal Wabbit library including C# interface Copyright (C) Microsoft Corp 2012-2016, Yahoo! Inc. 2007-2012, and many individual contributors. All rights reserved. vowpal-wabbit-8.5.0.dfsg1/cs/cs_json/packages.config000066400000000000000000000004701323713302100223110ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/cs_parallel/000077500000000000000000000000001323713302100201665ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/cs_parallel/Properties/000077500000000000000000000000001323713302100223225ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/cs_parallel/Properties/AssemblyInfo.cs000066400000000000000000000022351323713302100252460ustar00rootroot00000000000000//------------------------------------------------------------------------------ // // This code was generated by a tool. // Runtime Version:4.0.30319.42000 // // Changes to this file may cause incorrect behavior and will be lost if // the code is regenerated. // //------------------------------------------------------------------------------ [assembly: System.Reflection.AssemblyTitle("Vowpal Wabbit Parallel")] [assembly: System.Reflection.AssemblyDescription("Vowpal Wabbit Parallel")] [assembly: System.Reflection.AssemblyCompany("Microsoft Corp")] [assembly: System.Reflection.AssemblyProduct("Vowpal Wabbit")] [assembly: System.Reflection.AssemblyCopyright("Copyright (C) Microsoft Corp 2012-2016, Yahoo! Inc. 2007-2012, and many individua" + "l contributors. All rights reserved")] [assembly: System.Runtime.InteropServices.ComVisible(false)] [assembly: System.CLSCompliant(false)] [assembly: System.Runtime.InteropServices.Guid("0bb98c1a-b25f-43a0-94b6-fed77f7e5cd8")] [assembly: System.Reflection.AssemblyVersion("8.4.0.1")] [assembly: System.Reflection.AssemblyFileVersion("8.4.0.1")] vowpal-wabbit-8.5.0.dfsg1/cs/cs_parallel/VowpalWabbitAsync.cs000066400000000000000000000316061323713302100241220ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Diagnostics.Contracts; using System.Linq; using System.Text; using System.Threading.Tasks; using VW.Labels; using VW.Serializer; namespace VW { /// /// An async wrapper VW supporting data ingest using declarative serializer infrastructure used with . /// /// The user type to be serialized. public class VowpalWabbitAsync : IDisposable { /// /// The owning manager. /// private VowpalWabbitThreadedLearning manager; /// /// The serializers are not thread-safe. Thus we need to allocate one for each VW instance. /// private IVowpalWabbitSerializer[] serializers; internal VowpalWabbitAsync(VowpalWabbitThreadedLearning manager) { Contract.Requires(manager != null); Contract.Ensures(this.serializers != null); this.manager = manager; // create a serializer for each instance - maintaining separate example caches var serializer = VowpalWabbitSerializerFactory.CreateSerializer(manager.Settings); this.serializers = this.manager.VowpalWabbits .Select(vw => serializer.Create(vw)) .ToArray(); } /// /// Learns from the given example. /// /// The example to learn. /// The label for this . /// /// The method only enqueues the example for learning and returns immediately. /// You must not re-use the example. /// public void Learn(TExample example, ILabel label = null) { Contract.Requires(example != null); Contract.Requires(label != null); manager.Post(vw => { using (var ex = this.serializers[vw.Settings.Node].Serialize(example, label)) { ex.Learn(); } }); } /// /// Predicts for the given example. /// /// The example to predict for. /// /// The method only enqueues the example for prediction and returns immediately. /// You must not re-use the example. /// public void Predict(TExample example) { Contract.Requires(example != null); manager.Post(vw => { using (var ex = this.serializers[vw.Settings.Node].Serialize(example)) { ex.Predict(); } }); } /// /// Learns from the given example. /// /// The example to learn. /// The label for this . /// The prediction factory to be used. See . /// The prediction for the given . /// /// The method only enqueues the example for learning and returns immediately. /// Await the returned task to receive the prediction result. /// public Task Learn(TExample example, ILabel label, IVowpalWabbitPredictionFactory predictionFactory) { Contract.Requires(example != null); Contract.Requires(label != null); Contract.Requires(predictionFactory != null); return manager.Post(vw => { using (var ex = this.serializers[vw.Settings.Node].Serialize(example, label)) { return ex.Learn(predictionFactory); } }); } /// /// Predicts for the given example. /// /// The example to predict for. /// The prediction factory to be used. See . /// The prediction for the given . /// /// The method only enqueues the example for learning and returns immediately. /// Await the returned task to receive the prediction result. /// public Task Predict(TExample example, IVowpalWabbitPredictionFactory predictionFactory) { Contract.Requires(example != null); Contract.Requires(predictionFactory != null); return manager.Post(vw => { using (var ex = this.serializers[vw.Settings.Node].Serialize(example)) { return ex.Predict(predictionFactory); } }); } /// /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources. /// public void Dispose() { this.Dispose(true); GC.SuppressFinalize(this); } private void Dispose(bool disposing) { if (disposing) { if (this.serializers != null) { foreach (var serializer in this.serializers) { // free cached examples serializer.Dispose(); } this.serializers = null; } } } } /// /// An async VW wrapper for multiline ingest. /// /// The user type of the shared feature. /// The user type for each action dependent feature. public class VowpalWabbitAsync : IDisposable { /// /// The owning manager. /// private readonly VowpalWabbitThreadedLearning manager; /// /// The serializers are not thread-safe. Thus we need to allocate one for each VW instance. /// private VowpalWabbitSingleExampleSerializer[] serializers; /// /// The serializers are not thread-safe. Thus we need to allocate one for each VW instance. /// private VowpalWabbitSingleExampleSerializer[] actionDependentFeatureSerializers; internal VowpalWabbitAsync(VowpalWabbitThreadedLearning manager) { if (manager == null) throw new ArgumentNullException("manager"); if (manager.Settings == null) throw new ArgumentNullException("manager.Settings"); if (manager.Settings.ParallelOptions == null) throw new ArgumentNullException("manager.Settings.ParallelOptions"); if (manager.Settings.ParallelOptions.MaxDegreeOfParallelism <= 0) throw new ArgumentOutOfRangeException("MaxDegreeOfParallelism must be greater than zero."); Contract.Ensures(this.serializers != null); Contract.Ensures(this.actionDependentFeatureSerializers != null); Contract.EndContractBlock(); this.manager = manager; // create a serializer for each instance - maintaining separate example caches var serializer = VowpalWabbitSerializerFactory.CreateSerializer(manager.Settings) as VowpalWabbitSingleExampleSerializerCompiler; if (serializer == null) throw new ArgumentException(string.Format( "{0} maps to a multiline example. Use VowpalWabbitAsync<{0}> instead.", typeof(TExample))); var adfSerializer = VowpalWabbitSerializerFactory.CreateSerializer(manager.Settings) as VowpalWabbitSingleExampleSerializerCompiler; if (adfSerializer == null) throw new ArgumentException(string.Format( "{0} maps to a multiline example. Use VowpalWabbitAsync<{0}> instead.", typeof(TActionDependentFeature))); this.serializers = this.manager.VowpalWabbits .Select(vw => serializer.Create(vw)) .ToArray(); this.actionDependentFeatureSerializers = this.manager.VowpalWabbits .Select(vw => adfSerializer.Create(vw)) .ToArray(); } /// /// Learn from the given example and return the current prediction for it. /// /// The shared example. /// The action dependent features. /// The index of the example to learn within . /// The label for the example to learn. public void Learn(TExample example, IReadOnlyCollection actionDependentFeatures, int index, ILabel label) { Contract.Requires(example != null); Contract.Requires(actionDependentFeatures != null); Contract.Requires(index >= 0); Contract.Requires(label != null); manager.Post(vw => VowpalWabbitMultiLine.Learn( vw, this.serializers[vw.Settings.Node], this.actionDependentFeatureSerializers[vw.Settings.Node], example, actionDependentFeatures, index, label)); } /// /// Learn from the given example and return the current prediction for it. /// /// The shared example. /// The action dependent features. /// The index of the example to learn within . /// The label for the example to learn. /// The ranked prediction for the given examples. public Task[]> LearnAndPredict(TExample example, IReadOnlyCollection actionDependentFeatures, int index, ILabel label) { Contract.Requires(example != null); Contract.Requires(actionDependentFeatures != null); Contract.Requires(index >= 0); Contract.Requires(label != null); return manager.Post(vw => VowpalWabbitMultiLine.LearnAndPredict( vw, this.serializers[vw.Settings.Node], this.actionDependentFeatureSerializers[vw.Settings.Node], example, actionDependentFeatures, index, label)); } /// /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources. /// public void Dispose() { this.Dispose(true); GC.SuppressFinalize(this); } private void Dispose(bool disposing) { if (disposing) { if (this.serializers != null) { foreach (var serializer in this.serializers) { // free cached examples serializer.Dispose(); } this.serializers = null; } if (this.actionDependentFeatureSerializers != null) { foreach (var serializer in this.actionDependentFeatureSerializers) { // free cached examples serializer.Dispose(); } this.actionDependentFeatureSerializers = null; } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs_parallel/VowpalWabbitThreadedLearning.cs000066400000000000000000000432551323713302100262500ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using System; using System.Collections.Generic; using System.Diagnostics.Contracts; using System.Linq; using System.Threading; using System.Threading.Tasks; using System.Threading.Tasks.Dataflow; namespace VW { /// /// VW wrapper supporting multi-core learning by utilizing thread-based allreduce. /// public class VowpalWabbitThreadedLearning : IDisposable { /// /// Random generator used by uniform random example distributor. /// /// Initialized with static seed to enable reproducability. private readonly Random random = new Random(42); /// /// Configurable example distribution function choosing the vw instance for the next example. /// private readonly Func exampleDistributor; /// /// Native vw instances setup for thread-based allreduce /// private VowpalWabbit[] vws; /// /// Worker threads with a nice message queue infront that will start blocking once it's too full. /// private readonly ActionBlock>[] actionBlocks; /// /// The only offer non-blocking methods. Getting observers and calling OnNext() enables /// blocking once the queue is full. /// private readonly IObserver>[] observers; /// /// Invoked right after the root node performed AllReduce with the other instances. /// private readonly ConcurrentList> syncActions; /// /// Task enable waiting for clients on completion after all action blocks have finished (incl. cleanup). /// private Task[] completionTasks; /// /// Number of examples seen sofar. Used by round robin example distributor. /// private int exampleCount; /// /// Initializes a new instance of the class. /// /// Common settings used for vw instances. public VowpalWabbitThreadedLearning(VowpalWabbitSettings settings) { if (settings == null) throw new ArgumentNullException("settings"); if (settings.ParallelOptions == null) throw new ArgumentNullException("settings.ParallelOptions must be set"); Contract.EndContractBlock(); this.Settings = settings; if (this.Settings.ParallelOptions.CancellationToken == null) this.Settings.ParallelOptions.CancellationToken = new CancellationToken(); switch (this.Settings.ExampleDistribution) { case VowpalWabbitExampleDistribution.UniformRandom: this.exampleDistributor = _ => this.random.Next(this.observers.Length); break; case VowpalWabbitExampleDistribution.RoundRobin: this.exampleDistributor = localExampleCount => (int)(localExampleCount % this.observers.Length); break; } this.exampleCount = 0; this.syncActions = new ConcurrentList>(); this.vws = new VowpalWabbit[settings.ParallelOptions.MaxDegreeOfParallelism]; this.actionBlocks = new ActionBlock>[settings.ParallelOptions.MaxDegreeOfParallelism]; this.observers = new IObserver>[settings.ParallelOptions.MaxDegreeOfParallelism]; // setup AllReduce chain // root closure { var nodeSettings = (VowpalWabbitSettings)settings.Clone(); nodeSettings.Node = 0; var vw = this.vws[0] = new VowpalWabbit(nodeSettings); var actionBlock = this.actionBlocks[0] = new ActionBlock>( action => action(vw), new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = 1, TaskScheduler = settings.ParallelOptions.TaskScheduler, CancellationToken = settings.ParallelOptions.CancellationToken, BoundedCapacity = (int)settings.MaxExampleQueueLengthPerInstance }); } for (int i = 1; i < settings.ParallelOptions.MaxDegreeOfParallelism; i++) { // closure vars var nodeSettings = (VowpalWabbitSettings)settings.Clone(); nodeSettings.Root = this.vws[0]; nodeSettings.Node = (uint)i; var vw = this.vws[i] = new VowpalWabbit(nodeSettings); var actionBlock = this.actionBlocks[i] = new ActionBlock>( action => action(vw), new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = 1, TaskScheduler = settings.ParallelOptions.TaskScheduler, CancellationToken = settings.ParallelOptions.CancellationToken, BoundedCapacity = (int)settings.MaxExampleQueueLengthPerInstance }); } // get observers to allow for blocking calls this.observers = this.actionBlocks.Select(ab => ab.AsObserver()).ToArray(); this.completionTasks = new Task[settings.ParallelOptions.MaxDegreeOfParallelism]; // root closure { var vw = this.vws[0]; this.completionTasks[0] = this.actionBlocks[0].Completion .ContinueWith(_ => { // perform final AllReduce vw.EndOfPass(); // execute synchronization actions foreach (var syncAction in this.syncActions.RemoveAll()) { syncAction(vw); } }); } for (int i = 1; i < this.vws.Length; i++) { // perform final AllReduce var vw = this.vws[i]; this.completionTasks[i] = this.actionBlocks[i].Completion .ContinueWith(_ => vw.EndOfPass(), this.Settings.ParallelOptions.CancellationToken); } } /// /// VowpalWabbit instances participating in AllReduce. /// public VowpalWabbit[] VowpalWabbits { get { return this.vws; } } /// /// Creates a new instance of to feed examples of type . /// /// The user example type. /// A new instance of . public VowpalWabbitAsync Create() { return new VowpalWabbitAsync(this); } /// /// Creates a new instance of to feed multi-line /// examples of type and . /// /// The user example type. /// The user action dependent feature type. /// A new instance of . public VowpalWabbitAsync Create() { return new VowpalWabbitAsync(this); } /// /// Everytime examples have been enqueued, /// an AllReduce-sync operation () is injected. /// /// The number of examples enqueued so far. private uint CheckEndOfPass() { var exampleCount = (uint)Interlocked.Increment(ref this.exampleCount); // since there is no lock the input queue, it's not guaranteed that exactly // that number of examples are processed (but maybe a few more). if (exampleCount % this.Settings.ExampleCountPerRun == 0) { this.observers[0].OnNext(vw => { // perform AllReduce vw.EndOfPass(); // execute synchronization actions foreach (var syncAction in this.syncActions.RemoveAll()) { syncAction(vw); } }); for (int i = 1; i < this.observers.Length; i++) { // perform AllReduce this.observers[i].OnNext(vw => vw.EndOfPass()); } } return exampleCount; } /// /// Enqueues an action to be executed on one of vw instances. /// /// The action to be executed (e.g. Learn/Predict/...). /// If number of actions waiting to be executed has reached this method blocks. public void Post(Action action) { Contract.Requires(action != null); var exampleCount = this.CheckEndOfPass(); // dispatch this.observers[this.exampleDistributor(exampleCount)].OnNext(action); } /// /// Enqueues a task to be executed by single VowpalWabbit instance. /// /// Which VowpalWabbit instance chosen, is determined by . /// The return type of the task. /// The task to be executed. /// The awaitable result of the supplied task. internal Task Post(Func func) { Contract.Requires(func!= null); var exampleCount = this.CheckEndOfPass(); var completionSource = new TaskCompletionSource(); // dispatch to a Vowpal Wabbit instance this.observers[this.exampleDistributor(exampleCount)].OnNext(vw => { try { completionSource.SetResult(func(vw)); } catch (Exception ex) { completionSource.SetException(ex); } }); return completionSource.Task; } /// /// Learns from the given example. /// /// The example to learn. public void Learn(string line) { Contract.Requires(line != null); this.Post(vw => vw.Learn(line)); } /// /// Learns from the given example. /// /// The multi-line example to learn. public void Learn(IEnumerable lines) { Contract.Requires(lines != null); this.Post(vw => vw.Learn(lines)); } /// /// Synchronized performance statistics. /// /// The task is only completed after synchronization of all instances, triggered example. public Task PerformanceStatistics { get { var completionSource = new TaskCompletionSource(); this.syncActions.Add(vw => completionSource.SetResult(vw.PerformanceStatistics)); return completionSource.Task; } } /// /// Signal that no more examples are send. /// /// Task completes once the learning and cleanup is done. public Task Complete() { // make sure no more sync actions are added, which might otherwise never been called this.syncActions.CompleteAdding(); foreach (var actionBlock in this.actionBlocks) { actionBlock.Complete(); } return Task.WhenAll(this.completionTasks); } /// /// Saves a model as part of the synchronization. /// /// Task compeletes once the model is saved. public Task SaveModel() { var completionSource = new TaskCompletionSource(); this.syncActions.Add(vw => { vw.SaveModel(); completionSource.SetResult(true); }); return completionSource.Task; } /// /// Saves a model as part of the synchronization. /// /// Task compeletes once the model is saved. public Task SaveModel(string filename) { Contract.Requires(!string.IsNullOrEmpty(filename)); var completionSource = new TaskCompletionSource(); this.syncActions.Add(vw => { vw.SaveModel(filename); completionSource.SetResult(true); }); return completionSource.Task; } /// /// The settings shared across all instances. /// public VowpalWabbitSettings Settings { get; private set; } /// /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources. /// public void Dispose() { this.Dispose(true); GC.SuppressFinalize(this); } private void Dispose(bool disposing) { if (disposing) { if (this.completionTasks != null) { // mark completion this.Complete() .Wait(this.Settings.ParallelOptions.CancellationToken); // wait for all actionblocks to finish Task.WhenAll(this.completionTasks) .Wait(this.Settings.ParallelOptions.CancellationToken); this.completionTasks = null; } if (this.vws != null) { foreach (var vw in this.vws) { vw.Dispose(); } this.vws = null; } } } /// /// Thread-safe list implementation supporting completion. /// /// The element type. private class ConcurrentList { private bool completed = false; private readonly List items = new List(); private readonly object lockObject = new object(); /// /// Adds an object to the end of the list. /// /// The object to be added to the list. /// Throws an if the as called previously. public void Add(T item) { lock (this.lockObject) { if (completed) { throw new InvalidOperationException("ConcurrentList has been marked completed."); } this.items.Add(item); } } /// /// Marks this list as complete. Any subsequent calls to will trigger an . /// public void CompleteAdding() { lock (this.lockObject) { this.completed = true; } } /// /// Removes all elements from the list. /// /// The elements removed. public T[] RemoveAll() { lock (this.lockObject) { var ret = this.items.ToArray(); this.items.Clear(); return ret; } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs_parallel/cs_parallel.csproj000066400000000000000000000137411323713302100236770ustar00rootroot00000000000000 Debug AnyCPU {08636F79-5577-4AF2-8EED-EC8A5BC14AC4} Library Properties VW VowpalWabbit.Parallel v4.5.2 512 ..\..\vowpalwabbit\ true ..\..\vowpalwabbit\x64\Debug\ DEBUG;TRACE full x64 prompt MinimumRecommendedRules.ruleset ..\..\vowpalwabbit\x64\Debug\VowpalWabbit.Parallel.XML ..\..\vowpalwabbit\x64\Release\ TRACE true pdbonly x64 prompt MinimumRecommendedRules.ruleset ..\..\vowpalwabbit\x64\Release\VowpalWabbit.Parallel.XML true ..\vw_key.snk $(SolutionDir)\packages\gitlink.2.3.0\lib\net45\GitLink.exe True $(SolutionDir)\packages\Microsoft.Tpl.Dataflow.4.5.24\lib\portable-net45+win8+wpa81\System.Threading.Tasks.Dataflow.dll True {85e55ae0-3784-4968-9271-c81af560e1c1} vw_clr {e621e022-c1f8-433f-905a-ab9a3de072b7} vw_common {e4e962ae-7056-4eb0-a8c5-8dc824a4b068} cs This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. vowpal-wabbit-8.5.0.dfsg1/cs/cs_parallel/cs_parallel.nuspec000066400000000000000000000024551323713302100236740ustar00rootroot00000000000000 Vowpal Wabbit Parallel VowpalWabbit.Parallel vw vowpal wabbit langford ml machine learning John Langford et al $version$ https://github.com/JohnLangford/vowpal_wabbit/wiki/C%23-Binding https://github.com/JohnLangford/vowpal_wabbit/blob/master/LICENSE false Official Vowpal Wabbit library including C# interface Copyright (C) Microsoft Corp 2012-2016, Yahoo! Inc. 2007-2012, and many individual contributors. All rights reserved. vowpal-wabbit-8.5.0.dfsg1/cs/cs_parallel/packages.config000066400000000000000000000005001323713302100231260ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/cs_unittest_nofriend/000077500000000000000000000000001323713302100221355ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/cs_unittest_nofriend/Properties/000077500000000000000000000000001323713302100242715ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/cs_unittest_nofriend/Properties/AssemblyInfo.cs000066400000000000000000000026101323713302100272120ustar00rootroot00000000000000using System.Reflection; using System.Runtime.CompilerServices; using System.Runtime.InteropServices; // General Information about an assembly is controlled through the following // set of attributes. Change these attribute values to modify the information // associated with an assembly. [assembly: AssemblyTitle("cs_unittest_nofriend")] [assembly: AssemblyDescription("")] [assembly: AssemblyConfiguration("")] [assembly: AssemblyCompany("")] [assembly: AssemblyProduct("cs_unittest_nofriend")] [assembly: AssemblyCopyright("Copyright © 2016")] [assembly: AssemblyTrademark("")] [assembly: AssemblyCulture("")] // Setting ComVisible to false makes the types in this assembly not visible // to COM components. If you need to access a type in this assembly from // COM, set the ComVisible attribute to true on that type. [assembly: ComVisible(false)] // The following GUID is for the ID of the typelib if this project is exposed to COM [assembly: Guid("49a85b7e-197e-4265-90d0-886ea6c014f3")] // Version information for an assembly consists of the following four values: // // Major Version // Minor Version // Build Number // Revision // // You can specify all the values or you can default the Build and Revision Numbers // by using the '*' as shown below: // [assembly: AssemblyVersion("1.0.*")] [assembly: AssemblyVersion("1.0.0.0")] [assembly: AssemblyFileVersion("1.0.0.0")] vowpal-wabbit-8.5.0.dfsg1/cs/cs_unittest_nofriend/TestSerializer.cs000066400000000000000000000010241323713302100254320ustar00rootroot00000000000000using System; using Microsoft.VisualStudio.TestTools.UnitTesting; using VW.Serializer.Attributes; using VW; namespace cs_unittest_nofriend { class PrivateClass { [Feature] public int A { get; set; } } [TestClass] public class TestSerializer { [TestMethod] [ExpectedException(typeof(ArgumentException))] public void TestPrivateClassException() { using (var vw = new VowpalWabbit("")) { } } } } vowpal-wabbit-8.5.0.dfsg1/cs/cs_unittest_nofriend/cs_unittest_nofriend.csproj000066400000000000000000000113331323713302100276100ustar00rootroot00000000000000 Debug AnyCPU {49A85B7E-197E-4265-90D0-886EA6C014F3} Library Properties cs_unittest_nofriend cs_unittest_nofriend v4.5.2 512 {3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC} 10.0 $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion) $(ProgramFiles)\Common Files\microsoft shared\VSTT\$(VisualStudioVersion)\UITestExtensionPackages False UnitTest true full false bin\Debug\ DEBUG;TRACE prompt 4 pdbonly true bin\Release\ TRACE prompt 4 {85e55ae0-3784-4968-9271-c81af560e1c1} vw_clr {e621e022-c1f8-433f-905a-ab9a3de072b7} vw_common {e4e962ae-7056-4eb0-a8c5-8dc824a4b068} cs False False False False vowpal-wabbit-8.5.0.dfsg1/cs/leaktest/000077500000000000000000000000001323713302100175215ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/leaktest/Properties/000077500000000000000000000000001323713302100216555ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/leaktest/Properties/AssemblyInfo.cs000066400000000000000000000025661323713302100246100ustar00rootroot00000000000000using System.Reflection; using System.Runtime.CompilerServices; using System.Runtime.InteropServices; // General Information about an assembly is controlled through the following // set of attributes. Change these attribute values to modify the information // associated with an assembly. [assembly: AssemblyTitle("cs_leaktest")] [assembly: AssemblyDescription("")] [assembly: AssemblyConfiguration("")] [assembly: AssemblyCompany("")] [assembly: AssemblyProduct("cs_leaktest")] [assembly: AssemblyCopyright("Copyright © 2015")] [assembly: AssemblyTrademark("")] [assembly: AssemblyCulture("")] // Setting ComVisible to false makes the types in this assembly not visible // to COM components. If you need to access a type in this assembly from // COM, set the ComVisible attribute to true on that type. [assembly: ComVisible(false)] // The following GUID is for the ID of the typelib if this project is exposed to COM [assembly: Guid("d5a87745-39ce-427d-8463-5dde8d630f26")] // Version information for an assembly consists of the following four values: // // Major Version // Minor Version // Build Number // Revision // // You can specify all the values or you can default the Build and Revision Numbers // by using the '*' as shown below: // [assembly: AssemblyVersion("1.0.*")] [assembly: AssemblyVersion("1.0.0.0")] [assembly: AssemblyFileVersion("1.0.0.0")] vowpal-wabbit-8.5.0.dfsg1/cs/leaktest/TestLeak.cs000066400000000000000000000016621323713302100215710ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; namespace cs_leaktest { [TestClass] public class TestLeakClass : TestWrappedBase { #if DEBUG /// /// Tests if the leak detector actually works. /// /// Only possible in debug as VLD is not linked against in release. [TestMethod] public void TestLeak() { try { Run("cs_unittest.TestLeakClass", "Leak"); } catch (AssertFailedException ex) { Assert.IsTrue(ex.Message.Contains("Total 492 bytes")); // 123 *4 } } [TestMethod] public void TestNoLeak() { Run("cs_unittest.TestLeakClass", "NoLeak"); } #endif } } vowpal-wabbit-8.5.0.dfsg1/cs/leaktest/TestWrapped.cs000066400000000000000000001231171323713302100223170ustar00rootroot00000000000000 using Microsoft.VisualStudio.TestTools.UnitTesting; using cs_leaktest; namespace cs_unittest { [TestClass] public class TestWrapped : TestWrappedBase { [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test1() { Run("cs_unittest.RunTests", "CommandLine_Test1"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test2() { Run("cs_unittest.RunTests", "CommandLine_Test2"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test3() { Run("cs_unittest.RunTests", "CommandLine_Test3"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test4() { Run("cs_unittest.RunTests", "CommandLine_Test4"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test5() { Run("cs_unittest.RunTests", "CommandLine_Test5"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test6() { Run("cs_unittest.RunTests", "CommandLine_Test6"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test7() { Run("cs_unittest.RunTests", "CommandLine_Test7"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test8() { Run("cs_unittest.RunTests", "CommandLine_Test8"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test11() { Run("cs_unittest.RunTests", "CommandLine_Test11"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test12() { Run("cs_unittest.RunTests", "CommandLine_Test12"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test15() { Run("cs_unittest.RunTests", "CommandLine_Test15"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test21() { Run("cs_unittest.RunTests", "CommandLine_Test21"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test22() { Run("cs_unittest.RunTests", "CommandLine_Test22"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test23() { Run("cs_unittest.RunTests", "CommandLine_Test23"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test27() { Run("cs_unittest.RunTests", "CommandLine_Test27"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test28() { Run("cs_unittest.RunTests", "CommandLine_Test28"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test29() { Run("cs_unittest.RunTests", "CommandLine_Test29"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test30() { Run("cs_unittest.RunTests", "CommandLine_Test30"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test35() { Run("cs_unittest.RunTests", "CommandLine_Test35"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test36() { Run("cs_unittest.RunTests", "CommandLine_Test36"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test37() { Run("cs_unittest.RunTests", "CommandLine_Test37"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test38() { Run("cs_unittest.RunTests", "CommandLine_Test38"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test39() { Run("cs_unittest.RunTests", "CommandLine_Test39"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test40() { Run("cs_unittest.RunTests", "CommandLine_Test40"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test41() { Run("cs_unittest.RunTests", "CommandLine_Test41"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test62() { Run("cs_unittest.RunTests", "CommandLine_Test62"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test63() { Run("cs_unittest.RunTests", "CommandLine_Test63"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test64() { Run("cs_unittest.RunTests", "CommandLine_Test64"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test72() { Run("cs_unittest.RunTests", "CommandLine_Test72"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test73() { Run("cs_unittest.RunTests", "CommandLine_Test73"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test74() { Run("cs_unittest.RunTests", "CommandLine_Test74"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test75() { Run("cs_unittest.RunTests", "CommandLine_Test75"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test76() { Run("cs_unittest.RunTests", "CommandLine_Test76"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test78() { Run("cs_unittest.RunTests", "CommandLine_Test78"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test79() { Run("cs_unittest.RunTests", "CommandLine_Test79"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test80() { Run("cs_unittest.RunTests", "CommandLine_Test80"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test81() { Run("cs_unittest.RunTests", "CommandLine_Test81"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test82() { Run("cs_unittest.RunTests", "CommandLine_Test82"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test83() { Run("cs_unittest.RunTests", "CommandLine_Test83"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test88() { Run("cs_unittest.RunTests", "CommandLine_Test88"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test89() { Run("cs_unittest.RunTests", "CommandLine_Test89"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test90() { Run("cs_unittest.RunTests", "CommandLine_Test90"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test91() { Run("cs_unittest.RunTests", "CommandLine_Test91"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test94() { Run("cs_unittest.RunTests", "CommandLine_Test94"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test95() { Run("cs_unittest.RunTests", "CommandLine_Test95"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test97() { Run("cs_unittest.RunTests", "CommandLine_Test97"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test107() { Run("cs_unittest.RunTests", "CommandLine_Test107"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test108() { Run("cs_unittest.RunTests", "CommandLine_Test108"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test109() { Run("cs_unittest.RunTests", "CommandLine_Test109"); } [TestCategory("Command Line")] [TestMethod] public void CommandLine_Test113() { Run("cs_unittest.RunTests", "CommandLine_Test113"); } [TestMethod] public void TestAllReduce() { Run("cs_unittest.TestAllReduceClass", "TestAllReduce"); } [TestMethod] public void TestExampleCacheForLearning() { Run("cs_unittest.TestExampleCacheCases", "TestExampleCacheForLearning"); } [TestMethod] public void TestExampleCacheDisabledForLearning() { Run("cs_unittest.TestExampleCacheCases", "TestExampleCacheDisabledForLearning"); } [TestMethod] public void TestExampleCache() { Run("cs_unittest.TestExampleCacheCases", "TestExampleCache"); } [TestMethod] public void TestHash() { Run("cs_unittest.TestManagedHash", "TestHash"); } [TestCategory("Marshal")] [TestMethod] public void TestEnumerize() { Run("cs_unittest.TestMarshalling", "TestEnumerize"); } [TestCategory("Marshal")] [TestMethod] public void TestString() { Run("cs_unittest.TestMarshalling", "TestString"); } [TestCategory("Marshal")] [TestMethod] public void TestStringFeatureGroup() { Run("cs_unittest.TestMarshalling", "TestStringFeatureGroup"); } [TestCategory("Marshal")] [TestMethod] public void TestStringNamespace() { Run("cs_unittest.TestMarshalling", "TestStringNamespace"); } [TestCategory("Marshal")] [TestMethod] public void TestStringEscape() { Run("cs_unittest.TestMarshalling", "TestStringEscape"); } [TestCategory("Marshal")] [TestMethod] public void TestStringSplit() { Run("cs_unittest.TestMarshalling", "TestStringSplit"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionary() { Run("cs_unittest.TestMarshalling", "TestDictionary"); } [TestCategory("Marshal")] [TestMethod] public void TestCustomType() { Run("cs_unittest.TestMarshalling", "TestCustomType"); } [TestCategory("Marshal")] [TestMethod] public void TestEnumerableString() { Run("cs_unittest.TestMarshalling", "TestEnumerableString"); } [TestCategory("Marshal")] [TestMethod] public void TestEnumerableKV() { Run("cs_unittest.TestMarshalling", "TestEnumerableKV"); } [TestCategory("Marshal")] [TestMethod] public void TestComplexType() { Run("cs_unittest.TestMarshalling", "TestComplexType"); } [TestCategory("Marshal")] [TestMethod] public void TestEnumerizePosition() { Run("cs_unittest.TestMarshalling", "TestEnumerizePosition"); } [TestCategory("Marshal")] [TestMethod] public void TestBool() { Run("cs_unittest.TestMarshalling", "TestBool"); } [TestCategory("Marshal")] [TestMethod] public void TestFeatureDiscoveryAll() { Run("cs_unittest.TestMarshalling", "TestFeatureDiscoveryAll"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericInt64Overflow() { Run("cs_unittest.TestMarshallingOverflow", "TestNumericInt64Overflow"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericUInt64Overflow() { Run("cs_unittest.TestMarshallingOverflow", "TestNumericUInt64Overflow"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericDoubleOverflow() { Run("cs_unittest.TestMarshallingOverflow", "TestNumericDoubleOverflow"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericInt64OverflowArray() { Run("cs_unittest.TestMarshallingOverflow", "TestNumericInt64OverflowArray"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericUInt64OverflowArray() { Run("cs_unittest.TestMarshallingOverflow", "TestNumericUInt64OverflowArray"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericDoubleOverflowArray() { Run("cs_unittest.TestMarshallingOverflow", "TestNumericDoubleOverflowArray"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryUInt16UInt32() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryUInt16UInt32"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryUInt16Single() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryUInt16Single"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryUInt16Int64() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryUInt16Int64"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryUInt16UInt64() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryUInt16UInt64"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryUInt16Double() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryUInt16Double"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericByte() { Run("cs_unittest.TestMarshalNumeric", "TestNumericByte"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericByteArray() { Run("cs_unittest.TestMarshalNumeric", "TestNumericByteArray"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericByteArrayAnchor() { Run("cs_unittest.TestMarshalNumeric", "TestNumericByteArrayAnchor"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericSByte() { Run("cs_unittest.TestMarshalNumeric", "TestNumericSByte"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericSByteArray() { Run("cs_unittest.TestMarshalNumeric", "TestNumericSByteArray"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericSByteArrayAnchor() { Run("cs_unittest.TestMarshalNumeric", "TestNumericSByteArrayAnchor"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericInt16() { Run("cs_unittest.TestMarshalNumeric", "TestNumericInt16"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericInt16Array() { Run("cs_unittest.TestMarshalNumeric", "TestNumericInt16Array"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericInt16ArrayAnchor() { Run("cs_unittest.TestMarshalNumeric", "TestNumericInt16ArrayAnchor"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericInt32() { Run("cs_unittest.TestMarshalNumeric", "TestNumericInt32"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericInt32Array() { Run("cs_unittest.TestMarshalNumeric", "TestNumericInt32Array"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericInt32ArrayAnchor() { Run("cs_unittest.TestMarshalNumeric", "TestNumericInt32ArrayAnchor"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericUInt16() { Run("cs_unittest.TestMarshalNumeric", "TestNumericUInt16"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericUInt16Array() { Run("cs_unittest.TestMarshalNumeric", "TestNumericUInt16Array"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericUInt16ArrayAnchor() { Run("cs_unittest.TestMarshalNumeric", "TestNumericUInt16ArrayAnchor"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericUInt32() { Run("cs_unittest.TestMarshalNumeric", "TestNumericUInt32"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericUInt32Array() { Run("cs_unittest.TestMarshalNumeric", "TestNumericUInt32Array"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericUInt32ArrayAnchor() { Run("cs_unittest.TestMarshalNumeric", "TestNumericUInt32ArrayAnchor"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericSingle() { Run("cs_unittest.TestMarshalNumeric", "TestNumericSingle"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericSingleArray() { Run("cs_unittest.TestMarshalNumeric", "TestNumericSingleArray"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericSingleArrayAnchor() { Run("cs_unittest.TestMarshalNumeric", "TestNumericSingleArrayAnchor"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericInt64() { Run("cs_unittest.TestMarshalNumeric", "TestNumericInt64"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericInt64Array() { Run("cs_unittest.TestMarshalNumeric", "TestNumericInt64Array"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericInt64ArrayAnchor() { Run("cs_unittest.TestMarshalNumeric", "TestNumericInt64ArrayAnchor"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericUInt64() { Run("cs_unittest.TestMarshalNumeric", "TestNumericUInt64"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericUInt64Array() { Run("cs_unittest.TestMarshalNumeric", "TestNumericUInt64Array"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericUInt64ArrayAnchor() { Run("cs_unittest.TestMarshalNumeric", "TestNumericUInt64ArrayAnchor"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericDouble() { Run("cs_unittest.TestMarshalNumeric", "TestNumericDouble"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericDoubleArray() { Run("cs_unittest.TestMarshalNumeric", "TestNumericDoubleArray"); } [TestCategory("Marshal")] [TestMethod] public void TestNumericDoubleArrayAnchor() { Run("cs_unittest.TestMarshalNumeric", "TestNumericDoubleArrayAnchor"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryByte() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryByte"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryByteString() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryByteString"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryByteByte() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryByteByte"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryByteSByte() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryByteSByte"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryByteInt16() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryByteInt16"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryByteInt32() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryByteInt32"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryByteUInt16() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryByteUInt16"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryByteUInt32() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryByteUInt32"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryByteSingle() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryByteSingle"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryByteInt64() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryByteInt64"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryByteUInt64() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryByteUInt64"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryByteDouble() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryByteDouble"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionarySByte() { Run("cs_unittest.TestMarshalNumeric", "TestDictionarySByte"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionarySByteString() { Run("cs_unittest.TestMarshalNumeric", "TestDictionarySByteString"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionarySByteByte() { Run("cs_unittest.TestMarshalNumeric", "TestDictionarySByteByte"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionarySByteSByte() { Run("cs_unittest.TestMarshalNumeric", "TestDictionarySByteSByte"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionarySByteInt16() { Run("cs_unittest.TestMarshalNumeric", "TestDictionarySByteInt16"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionarySByteInt32() { Run("cs_unittest.TestMarshalNumeric", "TestDictionarySByteInt32"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionarySByteUInt16() { Run("cs_unittest.TestMarshalNumeric", "TestDictionarySByteUInt16"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionarySByteUInt32() { Run("cs_unittest.TestMarshalNumeric", "TestDictionarySByteUInt32"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionarySByteSingle() { Run("cs_unittest.TestMarshalNumeric", "TestDictionarySByteSingle"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionarySByteInt64() { Run("cs_unittest.TestMarshalNumeric", "TestDictionarySByteInt64"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionarySByteUInt64() { Run("cs_unittest.TestMarshalNumeric", "TestDictionarySByteUInt64"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionarySByteDouble() { Run("cs_unittest.TestMarshalNumeric", "TestDictionarySByteDouble"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt16() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt16"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt16String() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt16String"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt16Byte() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt16Byte"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt16SByte() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt16SByte"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt16Int16() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt16Int16"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt16Int32() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt16Int32"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt16UInt16() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt16UInt16"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt16UInt32() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt16UInt32"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt16Single() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt16Single"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt16Int64() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt16Int64"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt16UInt64() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt16UInt64"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt16Double() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt16Double"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt32() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt32"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt32String() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt32String"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt32Byte() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt32Byte"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt32SByte() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt32SByte"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt32Int16() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt32Int16"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt32Int32() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt32Int32"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt32UInt16() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt32UInt16"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt32UInt32() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt32UInt32"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt32Single() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt32Single"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt32Int64() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt32Int64"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt32UInt64() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt32UInt64"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryInt32Double() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryInt32Double"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryUInt16() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryUInt16"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryUInt16String() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryUInt16String"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryUInt16Byte() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryUInt16Byte"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryUInt16SByte() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryUInt16SByte"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryUInt16Int16() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryUInt16Int16"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryUInt16Int32() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryUInt16Int32"); } [TestCategory("Marshal")] [TestMethod] public void TestDictionaryUInt16UInt16() { Run("cs_unittest.TestMarshalNumeric", "TestDictionaryUInt16UInt16"); } [TestCategory("Model Loading")] [TestMethod] public void TestLoadModelCorrupt() { Run("cs_unittest.TestModelLoading", "TestLoadModelCorrupt"); } [TestCategory("Model Loading")] [TestMethod] public void TestLoadModel() { Run("cs_unittest.TestModelLoading", "TestLoadModel"); } [TestCategory("Model Loading")] [TestMethod] public void TestLoadModelRandomCorrupt() { Run("cs_unittest.TestModelLoading", "TestLoadModelRandomCorrupt"); } [TestCategory("Model Loading")] [TestMethod] public void TestLoadModelInMemory() { Run("cs_unittest.TestModelLoading", "TestLoadModelInMemory"); } [TestCategory("Model Loading")] [TestMethod] public void TestID() { Run("cs_unittest.TestModelLoading", "TestID"); } [TestCategory("Model Loading")] [TestMethod] public void TestReload() { Run("cs_unittest.TestModelLoading", "TestReload"); } [TestCategory("Null")] [TestMethod] public void TestNull1() { Run("cs_unittest.TestNull", "TestNull1"); } [TestCategory("Null")] [TestMethod] public void TestNull2() { Run("cs_unittest.TestNull", "TestNull2"); } [TestCategory("Null")] [TestMethod] public void TestNull3() { Run("cs_unittest.TestNull", "TestNull3"); } [TestCategory("Null")] [TestMethod] public void TestNull4() { Run("cs_unittest.TestNull", "TestNull4"); } [TestCategory("Null")] [TestMethod] public void TestNull5() { Run("cs_unittest.TestNull", "TestNull5"); } [TestMethod] public void TestCustomFeaturizer() { Run("cs_unittest.TestSerializer", "TestCustomFeaturizer"); } [TestMethod] public void TestCustomFeaturizerOverideMethod() { Run("cs_unittest.TestSerializer", "TestCustomFeaturizerOverideMethod"); } [TestMethod] public void TestDictify() { Run("cs_unittest.TestSerializer", "TestDictify"); } [TestCategory("Command line through marshalling")] [TestMethod] public void Test1and2() { Run("cs_test.Test1and2Class", "Test1and2"); } [TestCategory("Command line through marshalling")] [TestMethod] public void Test3() { Run("cs_unittest.Test3Class", "Test3"); } [TestCategory("Command line through marshalling")] [TestMethod] public void Test4and6() { Run("cs_unittest.Test3Class", "Test4and6"); } [TestCategory("Command line through marshalling")] [TestMethod] public void Test5() { Run("cs_unittest.Test3Class", "Test5"); } [TestCategory("Command line through marshalling")] [TestMethod] public void Test7and8() { Run("cs_unittest.Test3Class", "Test7and8"); } [TestCategory("Command line through marshalling")] [TestMethod] public void Test87() { Run("cs_unittest.TestCbAdfClass", "Test87"); } [TestMethod] public void TestSharedModel() { Run("cs_unittest.TestCbAdfClass", "TestSharedModel"); } [TestMethod] public void TestAntlr() { Run("cs_unittest.TestAntlrClass", "TestAntlr"); } [TestCategory("ObjectPool")] [TestMethod] public void ObjectPoolTestEmptyFactory() { Run("cs_unittest.TestPooling", "ObjectPoolTestEmptyFactory"); } [TestCategory("ObjectPool")] [TestMethod] public void ObjectPoolTestDisposed1() { Run("cs_unittest.TestPooling", "ObjectPoolTestDisposed1"); } [TestCategory("ObjectPool")] [TestMethod] public void ObjectPoolTestDisposed2() { Run("cs_unittest.TestPooling", "ObjectPoolTestDisposed2"); } [TestCategory("ObjectPool")] [TestMethod] public void ObjectPoolTestDangling() { Run("cs_unittest.TestPooling", "ObjectPoolTestDangling"); } [TestCategory("ObjectPool")] [TestMethod] public void ObjectPoolTestFactory() { Run("cs_unittest.TestPooling", "ObjectPoolTestFactory"); } [TestCategory("ObjectPool")] [TestMethod] public void ObjectPoolTestConcurrency() { Run("cs_unittest.TestPooling", "ObjectPoolTestConcurrency"); } [TestMethod] public void VwCleanupTest() { Run("cs_unittest.TestWrapper", "VwCleanupTest"); } [TestMethod] public void VwCleanupTestError() { Run("cs_unittest.TestWrapper", "VwCleanupTestError"); } [TestMethod] public void VwModelRefCountingTest() { Run("cs_unittest.TestWrapper", "VwModelRefCountingTest"); } } } vowpal-wabbit-8.5.0.dfsg1/cs/leaktest/TestWrappedBase.cs000066400000000000000000000077621323713302100231210ustar00rootroot00000000000000using cs_testcommon; using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections.Generic; using System.Diagnostics; using System.IO; using System.Linq; using System.Runtime.InteropServices; using System.Text; using System.Text.RegularExpressions; using System.Threading; using System.Threading.Tasks; using VLD; namespace cs_leaktest { public class TestWrappedBase { [DllImport("kernel32", SetLastError = true, CharSet = CharSet.Ansi)] static extern IntPtr LoadLibrary([MarshalAs(UnmanagedType.LPStr)]string lpFileName); [DllImport("kernel32.dll", SetLastError = true)] [return: MarshalAs(UnmanagedType.Bool)] static extern bool FreeLibrary(IntPtr hModule); /// /// Implement custom StackTrace so one can easily click in Test Explorer /// internal class CustomException : Exception { private readonly string stackTrace; internal CustomException(string message, string stackTrace) : base(message) { this.stackTrace = stackTrace; } public override string StackTrace { get { return this.stackTrace; } } } public TestContext TestContext { get; set; } private object lockObject = new object(); protected void Run(string type, string method) { lock (lockObject) { using (var vld = new VisualLeakDetector()) { try { // vowpalwabbit\x64\Debug\cs_leaktest.dll var basePath = Path.GetDirectoryName(typeof(VisualLeakDetector).Assembly.Location); var handle = LoadLibrary(basePath + @"\\VowpalWabbitCore.dll"); var appDomain = AppDomain.CreateDomain("Test1"); try { ITestRunner test1 = (ITestRunner)appDomain.CreateInstanceFromAndUnwrap(basePath + @"\\cs_unittest.dll", "cs_unittest.TestRunner"); Environment.CurrentDirectory = basePath + @"\..\..\..\test"; var result = test1.Run(type, method); if (result != null) { // check for exception marker var index = result.IndexOf("#-#-#-#-#-#-#"); if (index == -1) { Assert.Fail(result); } throw new CustomException(result.Substring(0, index), result.Substring(index + 13)); } } finally { AppDomain.Unload(appDomain); } try { FreeLibrary(handle); FreeLibrary(handle); vld.ReportLeaks(); var message = string.Concat(vld.Messages.Select(t => t.Item2)); var blocks = message.Split(new[] { "---------- Block " }, StringSplitOptions.None) .Where(block => Regex.IsMatch(block, "^\\d+ at")) .ToList(); Assert.AreEqual(0, blocks.Count, string.Join("\n", blocks)); } finally { LoadLibrary(basePath + @"\VowpalWabbitCore.dll"); } } finally { vld.MarkAllLeaksAsReported(); } } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/leaktest/cs_leaktest.csproj000066400000000000000000000160011323713302100232420ustar00rootroot00000000000000 Debug AnyCPU {492EA3A7-8A41-459E-BBC3-5A3FE9DDFC68} Library Properties cs_leaktest cs_leaktest v4.5.2 512 {3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC} 10.0 $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion) $(ProgramFiles)\Common Files\microsoft shared\VSTT\$(VisualStudioVersion)\UITestExtensionPackages False UnitTest ..\..\vowpalwabbit\ fc8dceef true $(SolutionDir)\x64\Debug\ DEBUG;TRACE true full x64 prompt MinimumRecommendedRules.ruleset $(SolutionDir)\x64\Release\ TRACE true pdbonly x64 prompt MinimumRecommendedRules.ruleset true $(SolutionDir)\x64\DebugLeakCheck\ DEBUG;TRACE true full x64 prompt MinimumRecommendedRules.ruleset true {6a9cbeab-427f-4d8d-9559-b76b42b0895c} cs_testcommon {3d57a6af-de8c-40dc-abde-e4ce1b9c0d20} cs_vld dbghelp.dll PreserveNewest vld_x64.dll PreserveNewest Microsoft.DTfW.DHL.manifest PreserveNewest False False False False This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. vowpal-wabbit-8.5.0.dfsg1/cs/setup/000077500000000000000000000000001323713302100170455ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/setup/Product.wxs000066400000000000000000000055121323713302100212330ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/setup/packages.config000066400000000000000000000002021323713302100220040ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/setup/setup.wixproj000066400000000000000000000147451323713302100216440ustar00rootroot00000000000000 Debug x86 3.10 1f25d39d-71c3-4fde-a58f-d3d5d4f6dffb 2.0 VowpalWabbit Package $(MSBuildExtensionsPath32)\Microsoft\WiX\v3.x\Wix.targets $(MSBuildExtensionsPath)\Microsoft\WiX\v3.x\Wix.targets ..\..\vowpalwabbit\ ..\..\vowpalwabbit\x64\$(Configuration)\ obj\$(Configuration)\ Debug ..\..\vowpalwabbit\x64\$(Configuration)\ obj\$(Configuration)\ Debug ..\..\vowpalwabbit\x64\$(Configuration)\ obj\$(Platform)\$(Configuration)\ ..\..\vowpalwabbit\x64\$(Configuration)\ obj\$(Platform)\$(Configuration)\ True vw {1055a78f-1e3a-4e6c-bbf5-0b63299c4adf} True True Binaries;Content;Satellites INSTALLFOLDER vw_clr {85e55ae0-3784-4968-9271-c81af560e1c1} True True Binaries;Content;Satellites INSTALLFOLDER vw_common {e621e022-c1f8-433f-905a-ab9a3de072b7} True True Binaries;Content;Satellites INSTALLFOLDER cs {e4e962ae-7056-4eb0-a8c5-8dc824a4b068} True True Binaries;Content;Satellites INSTALLFOLDER cs_console {01a85382-c3e9-480a-86bf-fafe4ad107a7} True True Binaries;Content;Satellites INSTALLFOLDER cs_json {9e27fa94-ab34-4736-8427-fb7a2ba90d52} True True Binaries;Content;Satellites INSTALLFOLDER cs_parallel {08636f79-5577-4af2-8eed-ec8a5bc14ac4} True True Binaries;Content;Satellites INSTALLFOLDER This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. vowpal-wabbit-8.5.0.dfsg1/cs/setup_bundle/000077500000000000000000000000001323713302100203765ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/setup_bundle/Product.wxs000066400000000000000000000027671323713302100225750ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/setup_bundle/packages.config000066400000000000000000000002021323713302100233350ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/setup_bundle/setup_bundle.wixproj000066400000000000000000000124401323713302100245140ustar00rootroot00000000000000 Debug x86 3.10 a80650fd-7dca-4b80-b52e-d22d5cae9e92 2.0 vowpalwabbit_setup Bundle $(MSBuildExtensionsPath32)\Microsoft\WiX\v3.x\Wix.targets $(MSBuildExtensionsPath)\Microsoft\WiX\v3.x\Wix.targets ..\..\vowpalwabbit\ bin\$(Configuration)\ obj\$(Configuration)\ Debug ..\..\vowpalwabbit\x64\$(Configuration)\ obj\$(Configuration)\ Debug ..\..\vowpalwabbit\x64\$(Configuration)\ obj\$(Platform)\$(Configuration)\ ..\..\vowpalwabbit\x64\$(Configuration)\ obj\$(Platform)\$(Configuration)\ Debug bin\$(Platform)\$(Configuration)\ obj\$(Platform)\$(Configuration)\ bin\$(Platform)\$(Configuration)\ obj\$(Platform)\$(Configuration)\ Debug bin\$(Platform)\$(Configuration)\ obj\$(Platform)\$(Configuration)\ bin\$(Platform)\$(Configuration)\ obj\$(Platform)\$(Configuration)\ setup {1f25d39d-71c3-4fde-a58f-d3d5d4f6dffb} True True Binaries;Content;Satellites INSTALLFOLDER $(WixExtDir)\WixNetFxExtension.dll WixNetFxExtension $(WixExtDir)\WixBalExtension.dll WixBalExtension This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. vowpal-wabbit-8.5.0.dfsg1/cs/test/000077500000000000000000000000001323713302100166645ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/test/AttributesSample.cs000066400000000000000000000145051323713302100225100ustar00rootroot00000000000000using System; using System.Collections.Generic; using System.Linq; using System.Text; using VW.Serializer.Attributes; using VW.Serializer; using VW; using VW.Labels; namespace cs_test { public static class AttributesSample { public static void Attributes() { var d1 = new DocumentFeature { Id = "d1", Time = new DateTime(2015, 1, 1), Value = new LDAFeatureVector { Values = new[] { 1.0, 2.0, 3.0 } } }; var context = new UserContext { User = new UserFeature { Age = Age.Adult, Gender = Gender.Female, Location = "WA", FeatureBag = new Dictionary { { "Foo", 1.1f }, { "Bar", 2.0f } } }, UserLDATopicPreference = new LDAFeatureVector { Values = new[] { 0.1, 0.2, 0.3 } }, ActionDependentFeatures = new List { d1, new DocumentFeature { Id = "d2", Time = new DateTime(2015,1,1), Value = new LDAFeatureVector { Values = new [] { 1.0, 2.0, 3.0 } } }, d1 } }; //var visitor = new VowpalWabbitStringVisitor(); //var serializer = VowpalWabbitSerializerFactory.CreateSerializer(visitor); //var serializerDependent = VowpalWabbitSerializerFactory.CreateSerializer(visitor); //Console.WriteLine(serializer.Serialize(context)); //foreach (var actionDependentFeature in context.ActionDependentFeatures) //{ // Console.WriteLine(serializerDependent.Serialize(actionDependentFeature)); //} //using (var pool = new ObjectPool>(() => new VowpalWabbit(""))) //{ // using (var vw = pool.Get()) // { // // do work with VW // // vw.Value.CreateEmptyExample(); // } // // don't modify this model from another thread! // var newVwModel = new VowpalWabbitModel("model init"); // pool.UpdateFactory(() => new VowpalWabbit(newVwModel)); // // this will get a new VW instance with a newer version! // using (var vw = pool.Get()) // { // // do work with VW // // vw.Value.CreateEmptyExample(); // } //} Console.ReadKey(); } public static void RunFeaturesTest() { var context = new FeatureTestContext { S = new[] { "p^the_man", "w^thew^man\u0394", "w^man" }, T = new[] { "p^un_homme", "w^un", "w^homme" } }; var vw = new VowpalWabbit("-q st --noconstant --quiet"); vw.Native.Learn("1 |s p^the_man w^the w^man |t p^un_homme w^un w^homme"); var prediction = vw.Learn(context, new SimpleLabel { Label = 1f }, VowpalWabbitPredictionType.Scalar); Console.Error.WriteLine("p2 = {0}", prediction); } } public class FeatureTestContext { [Feature(FeatureGroup = 's')] public IEnumerable S { get; set; } [Feature(FeatureGroup = 't')] public IEnumerable T { get; set; } } public class UserContext { [Feature(Namespace = "otheruser", FeatureGroup = 'o')] public UserFeature User { get; set; } [Feature(Namespace = "userlda", FeatureGroup = 'u')] public LDAFeatureVector UserLDATopicPreference { get; set; } public IReadOnlyList ActionDependentFeatures { get; set; } } [Cacheable(EqualityComparer = typeof(DocumentFeatureEqualityComparer))] public class DocumentFeature { public string Id { get; set; } public string SomeOtherId { get; set; } public DateTime Time { get; set; } // If we include this, it would result in mixing dense and non-dense features. // [Feature] public string ContentProvider { get; set; } [Feature(Namespace = "doclda", FeatureGroup = 'd')] public LDAFeatureVector Value { get; set; } } public class DocumentFeatureEqualityComparer : IEqualityComparer { public bool Equals(DocumentFeature x, DocumentFeature y) { return x.Id == y.Id && x.Time == y.Time; // maybe compare the full vector - not so sure on this part though // x.Value.Zip(y.Value, (a, b) => a == b).All(c => c); } public int GetHashCode(DocumentFeature obj) { return obj.Id.GetHashCode() + obj.Time.GetHashCode(); } } public enum Gender { Female, Male, Unknown } public class LDAFeatureVector { public string Compressed { get; set; } private double[] values; [Feature] public double[] Values { get { if (this.Compressed == null) { // e.g. call into decompression } return this.values; } set { this.values = value; } } } public class UserFeature { [Feature] public Age Age { get; set; } [Feature] public int? Income { get; set; } [Feature] public Gender Gender { get; set; } [Feature] public string Location { get; set; } [Feature] public Dictionary FeatureBag { get; set; } [Feature] public DayOfWeek DayOfWeek { get; set; } /// /// Will generate 24 parameters /// [Feature(Enumerize = true)] public int HourOfDay { get; set; } } public enum Age { Child, Adult, Elderly } } vowpal-wabbit-8.5.0.dfsg1/cs/test/NIPS2015Tutorial.cs000066400000000000000000000063321323713302100220240ustar00rootroot00000000000000using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using VW; using VW.Labels; using VW.Serializer.Attributes; namespace cs_test { /// /// Code examples as demonstrated during NIPS 2015 Tutorial (slides: https://github.com/JohnLangford/vowpal_wabbit/wiki/Tutorial) /// public class NIPS2015Tutorial { public void BasicExample() { using (var vw = new VowpalWabbit("--quiet")) { vw.Learn("1 |f 13:3.9656971e-02 24:3.4781646e-02 69:4.6296168e-02"); var prediction = vw.Predict("|f 13:3.9656971e-02 24:3.4781646e-02 69:4.6296168e-02", VowpalWabbitPredictionType.Scalar); vw.SaveModel("output.model"); } } public class MyExample { [Feature(FeatureGroup = 'p')] public float Income { get; set; } [Feature(Enumerize = true)] public int Age { get; set; } } public static void AnnotationExample() { using (var vw = new VowpalWabbit(new VowpalWabbitSettings { EnableStringExampleGeneration = true })) { var ex = new MyExample { Income = 40, Age = 25 }; var label = new SimpleLabel { Label = 1 }; var str = vw.Serializer.Create(vw.Native).SerializeToString(ex, label); // 1 |p Income:4 | Age25 vw.Learn(ex, label); var prediction = vw.Predict(ex, VowpalWabbitPredictionType.Scalar); } } public static void MultiThreadedPrediction() { var example = new MyExample { Income = 40, Age = 25 }; var vwModel = new VowpalWabbitModel("-t -i m1.model"); using (var pool = new VowpalWabbitThreadedPrediction(vwModel)) { // thread-safe using (var vw = pool.GetOrCreate()) { // vw.Value is not thread-safe vw.Value.Predict(example); } // thread-safe pool.UpdateModel(new VowpalWabbitModel("-t -i m2.model")); } } public static async Task MultiThreadedLearning() { var example = new MyExample { Income = 40, Age = 25 }; var label = new SimpleLabel { Label = 1 }; var settings = new VowpalWabbitSettings { ParallelOptions = new ParallelOptions { MaxDegreeOfParallelism = 16 }, ExampleCountPerRun = 2000, ExampleDistribution = VowpalWabbitExampleDistribution.RoundRobin }; using (var vw = new VowpalWabbitThreadedLearning(settings)) { using (var vwManaged = vw.Create()) { var prediction = await vwManaged.Learn(example, label, VowpalWabbitPredictionType.Scalar); } var saveModelTask = vw.SaveModel("m1.model"); await vw.Complete(); await saveModelTask; } } } } vowpal-wabbit-8.5.0.dfsg1/cs/test/Program.cs000066400000000000000000000471071323713302100206330ustar00rootroot00000000000000using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.IO; using VW; using System.Runtime.InteropServices; using System.Diagnostics; using VW.Serializer.Attributes; using System.Globalization; using cs_unittest.cbadf; using System.Threading.Tasks; using cs_unittest; namespace cs_test { class Program { static void Main(string[] args) { NIPS2015Tutorial.AnnotationExample(); // AttributesSample.Attributes(); //AttributesSample.RunFeaturesTest(); //LabDemo.Run(); RunFeaturesTest(); RunParserTest(); RunSpeedTest(); RunFlatExampleTestEx(); //RunLDAPredict(); //RunVWParse_and_VWLearn(); } private static void RunFeaturesTest() { // this usually requires that the library script to update train.w or its moral equivalent needs to have been run IntPtr vw = VowpalWabbitInterface.Initialize("-q st --noconstant --quiet"); IntPtr example = VowpalWabbitInterface.ReadExample(vw, "1 |s p^the_man w^the w^man |t p^un_homme w^un w^homme"); float score = VowpalWabbitInterface.Learn(vw, example); VowpalWabbitInterface.FinishExample(vw, example); VowpalWabbitInterface.FEATURE_SPACE[] featureSpace = new VowpalWabbitInterface.FEATURE_SPACE[2];//maximum number of index spaces VowpalWabbitInterface.FEATURE[] sfeatures = new VowpalWabbitInterface.FEATURE[3];// the maximum number of features VowpalWabbitInterface.FEATURE[] tfeatures = new VowpalWabbitInterface.FEATURE[3];// the maximum number of features GCHandle pinnedsFeatures = GCHandle.Alloc(sfeatures, GCHandleType.Pinned); GCHandle pinnedtFeatures = GCHandle.Alloc(tfeatures, GCHandleType.Pinned); featureSpace[0].features = pinnedsFeatures.AddrOfPinnedObject(); featureSpace[1].features = pinnedtFeatures.AddrOfPinnedObject(); GCHandle pinnedFeatureSpace = GCHandle.Alloc(featureSpace, GCHandleType.Pinned); IntPtr featureSpacePtr = pinnedFeatureSpace.AddrOfPinnedObject(); uint snum = VowpalWabbitInterface.HashSpace("s"); featureSpace[0].name = (byte)'s'; sfeatures[0].weight_index = VowpalWabbitInterface.HashFeature("p^the_man", snum); sfeatures[0].x = 1; // add the character "delta" to test unicode // do it as a string to test the marshaling is doing pinning correctly. const string s = "w^thew^man\u0394"; sfeatures[1].weight_index = VowpalWabbitInterface.HashFeature(s, snum); sfeatures[1].x = 1; sfeatures[2].weight_index = VowpalWabbitInterface.HashFeature("w^man", snum); sfeatures[2].x = 1; featureSpace[0].len = 3; uint tnum = VowpalWabbitInterface.HashSpace("t"); featureSpace[1].name = (byte)'t'; tfeatures[0].weight_index = VowpalWabbitInterface.HashFeature("p^un_homme", tnum); tfeatures[0].x = 1; tfeatures[1].weight_index = VowpalWabbitInterface.HashFeature("w^un", tnum); tfeatures[1].x = 1; tfeatures[2].weight_index = VowpalWabbitInterface.HashFeature("w^homme", tnum); tfeatures[2].x = 1; featureSpace[1].len = 3; IntPtr importedExample = VowpalWabbitInterface.ImportExample(vw, featureSpacePtr, (IntPtr)featureSpace.Length); VowpalWabbitInterface.AddLabel(importedExample, 1); score = VowpalWabbitInterface.Learn(vw, importedExample); Console.Error.WriteLine("p2 = {0}", score); VowpalWabbitInterface.Finish(vw); // clean up the memory we allocated pinnedsFeatures.Free(); pinnedtFeatures.Free(); pinnedFeatureSpace.Free(); } private static void RunParserTest() { IntPtr vw = VowpalWabbitInterface.Initialize("-q st -d ..\\..\\..\\test\\train-sets\\0002.dat -f out"); VowpalWabbitInterface.StartParser(vw, false); int count = 0; IntPtr example = IntPtr.Zero; while (IntPtr.Zero != (example = VowpalWabbitInterface.GetExample(vw))) { float label = VowpalWabbitInterface.GetLabel(example); count++; IntPtr featureSpaceLen = (IntPtr)0; IntPtr featureSpacePtr = VowpalWabbitInterface.ExportExample(vw, example, ref featureSpaceLen); VowpalWabbitInterface.FEATURE_SPACE[] featureSpace = new VowpalWabbitInterface.FEATURE_SPACE[(int)featureSpaceLen]; int featureSpace_size = Marshal.SizeOf(typeof(VowpalWabbitInterface.FEATURE_SPACE)); for (int i = 0; i < (int)featureSpaceLen; i++) { IntPtr curfeatureSpacePos = new IntPtr(featureSpacePtr.ToInt32() + i * featureSpace_size); featureSpace[i] = (VowpalWabbitInterface.FEATURE_SPACE)Marshal.PtrToStructure(curfeatureSpacePos, typeof(VowpalWabbitInterface.FEATURE_SPACE)); VowpalWabbitInterface.FEATURE[] feature = new VowpalWabbitInterface.FEATURE[featureSpace[i].len]; int feature_size = Marshal.SizeOf(typeof(VowpalWabbitInterface.FEATURE)); for (int j = 0; j < featureSpace[i].len; j++) { IntPtr curfeaturePos = new IntPtr((featureSpace[i].features.ToInt32() + j * feature_size)); feature[j] = (VowpalWabbitInterface.FEATURE)Marshal.PtrToStructure(curfeaturePos, typeof(VowpalWabbitInterface.FEATURE)); } } VowpalWabbitInterface.ReleaseFeatureSpace(featureSpacePtr, featureSpaceLen); float score = VowpalWabbitInterface.Learn(vw, example); VowpalWabbitInterface.FinishExample(vw, example); } VowpalWabbitInterface.EndParser(vw); VowpalWabbitInterface.Finish(vw); } private static void RunSpeedTest() { Console.WriteLine(DateTime.Now.Millisecond + DateTime.Now.Second * 1000 + DateTime.Now.Minute * 60 * 1000); //IntPtr vw = VowpalWabbitInterface.Initialize("--ngram 2 --skips 4 -l 0.25 -b 22 -d rcv1.train.raw.txt -f out"); //IntPtr vw = VowpalWabbitInterface.Initialize("-d rcv1.train.raw.txt -b 22 --ngram 2 --skips 4 -l 0.25 -c"); //IntPtr vw = VowpalWabbitInterface.Initialize("-d rcv1.train.raw.txt -c"); IntPtr vw = VowpalWabbitInterface.Initialize("-d ..\\..\\..\\test\\train-sets\\0002.dat"); VowpalWabbitInterface.StartParser(vw, false); int count = 0; IntPtr example = IntPtr.Zero; Stopwatch s = Stopwatch.StartNew(); while (IntPtr.Zero != (example = VowpalWabbitInterface.GetExample(vw))) { count++; float score = VowpalWabbitInterface.Learn(vw, example); VowpalWabbitInterface.FinishExample(vw, example); } s.Stop(); long t1 = s.ElapsedMilliseconds; VowpalWabbitInterface.EndParser(vw); VowpalWabbitInterface.Finish(vw); Console.WriteLine(DateTime.Now.Millisecond + DateTime.Now.Second * 1000 + DateTime.Now.Minute * 60 * 1000); Debug.WriteLine("RunSpeedTest Elapsed Time: {0} ms", s.ElapsedMilliseconds); Console.WriteLine("RunSpeedTest Elapsed Time: {0} ms", s.ElapsedMilliseconds); } private static void RunFlatExampleTestEx() { //IntPtr vw = VowpalWabbitInterface.Initialize("-q st -d rcv1.train.raw.txt -f out"); IntPtr vw = VowpalWabbitInterface.Initialize("-q st -d ..\\..\\..\\test\\train-sets\\0002.dat -f out"); VowpalWabbitInterface.StartParser(vw, false); uint stride = (uint)VowpalWabbitInterface.Get_Stride(vw); int count = 0; IntPtr example = IntPtr.Zero; while (IntPtr.Zero != (example = VowpalWabbitInterface.GetExample(vw))) { count++; float prediction = VowpalWabbitInterface.GetPrediction(example); float importance = VowpalWabbitInterface.GetImportance(example); float initial = VowpalWabbitInterface.GetInitial(example); float label = VowpalWabbitInterface.GetLabel(example); UInt32 tag_len = (UInt32)VowpalWabbitInterface.GetTagLength(example); byte[] tag = new byte[tag_len]; if (tag_len > 0) Marshal.Copy(VowpalWabbitInterface.GetTag(example), tag, 0, (int)tag_len); UInt32 num_features = (UInt32)VowpalWabbitInterface.GetFeatureNumber(example); VowpalWabbitInterface.FEATURE[] f; if (num_features > 0) { f = new VowpalWabbitInterface.FEATURE[num_features]; IntPtr feature_count = (IntPtr)0; IntPtr ret = VowpalWabbitInterface.GetFeatures(vw, example, ref feature_count); int feature_size = Marshal.SizeOf(typeof(VowpalWabbitInterface.FEATURE)); for (int i = 0; i < (int)feature_count; i++) { IntPtr curfeaturePos = new IntPtr(ret.ToInt32() + i * feature_size); f[i] = (VowpalWabbitInterface.FEATURE)Marshal.PtrToStructure(curfeaturePos, typeof(VowpalWabbitInterface.FEATURE)); } } VowpalWabbitInterface.FinishExample(vw, example); } VowpalWabbitInterface.EndParser(vw); VowpalWabbitInterface.Finish(vw); } public class VWInstanceEx { public VowpalWabbitInterface.FEATURE_SPACE[] featureSpace; public VWInstanceEx(IntPtr vw, IntPtr ex) { if (IntPtr.Zero == vw || IntPtr.Zero == ex) return; IntPtr featureSpaceLen = (IntPtr)0; IntPtr featureSpacePtr = VowpalWabbitInterface.ExportExample(vw, ex, ref featureSpaceLen); this.featureSpace = new VowpalWabbitInterface.FEATURE_SPACE[(int)featureSpaceLen]; int featureSpace_size = Marshal.SizeOf(typeof(VowpalWabbitInterface.FEATURE_SPACE)); for (int i = 0; i < (int)featureSpaceLen; i++) { IntPtr curfeatureSpacePos = new IntPtr(featureSpacePtr.ToInt32() + i * featureSpace_size); this.featureSpace[i] = (VowpalWabbitInterface.FEATURE_SPACE)Marshal.PtrToStructure(curfeatureSpacePos, typeof(VowpalWabbitInterface.FEATURE_SPACE)); VowpalWabbitInterface.FEATURE[] feature = new VowpalWabbitInterface.FEATURE[this.featureSpace[i].len]; int feature_size = Marshal.SizeOf(typeof(VowpalWabbitInterface.FEATURE)); for (int j = 0; j < this.featureSpace[i].len; j++) { IntPtr curfeaturePos = new IntPtr((this.featureSpace[i].features.ToInt32() + j * feature_size)); feature[j] = (VowpalWabbitInterface.FEATURE)Marshal.PtrToStructure(curfeaturePos, typeof(VowpalWabbitInterface.FEATURE)); } } VowpalWabbitInterface.ReleaseFeatureSpace(featureSpacePtr, (IntPtr)featureSpaceLen); } } private static void RunLDAPredict() { IntPtr vw = VowpalWabbitInterface.Initialize("-i wiki1k.model -t --quiet"); IntPtr example = VowpalWabbitInterface.ReadExample(vw, "| 0:1 2049:6 2:3 5592:1 2796:1 6151:1 6154:1 6157:2 6160:2 1027:2 6168:1 4121:1 6170:1 4124:1 29:1 35:1 2088:1 2091:1 2093:2 2095:3 4145:3 5811:1 53:1 58:1 6204:6 66:2 69:2 4167:1 6216:2 75:3 2402:1 86:1 2135:2 3126:1 4185:1 90:4 2144:1 4193:1 99:1 7185:2 2156:1 110:2 2161:1 114:2 1043:1 2165:1 2166:3 119:2 6265:1 4222:3 4224:1 4230:1 705:1 2674:1 6287:1 2192:1 145:7 2198:1 2200:2 4263:1 6312:1 5148:1 4269:3 6320:4 2227:1 4283:1 4285:2 1397:2 197:2 2246:3 2247:12 201:1 4299:1 2253:1 6351:4 6353:1 4306:1 6179:1 212:1 215:3 2264:1 3108:1 2266:1 224:1 4321:1 6372:1 229:1 2281:4 6381:1 4336:1 241:2 6388:1 2294:1 2297:1 1066:1 6402:1 6405:1 6410:7 6412:2 2322:5 2329:2 282:2 6191:1 6428:1 6431:1 6433:1 4386:21 6436:5 4390:3 6439:3 296:3 1415:3 6444:3 2350:2 2354:5 307:1 6457:3 315:1 319:1 4416:4 4419:1 325:1 326:2 6472:1 6474:1 334:2 1421:2 2384:1 1516:1 340:1 4438:1 344:2 6492:5 2401:1 354:1 4452:2 6505:4 402:3 4463:1 2418:1 2451:3 375:1 4472:1 4478:2 4479:2 2437:2 4487:1 4489:2 4493:2 2448:1 5528:1 4498:1 6547:4 6549:1 406:2 2673:1 2456:2 6554:1 4507:1 4513:1 418:3 6563:1 6566:1 5873:1 2472:10 1095:1 6572:1 4525:1 4529:2 2485:2 4535:15 6587:1 444:3 6590:1 449:1 456:1 2509:6 6221:3 6562:1 2467:1 468:1 902:2 2519:1 2607:1 4653:1 6626:1 422:1 2539:6 493:4 494:1 4591:1 6644:2 3156:1 2554:1 509:1 4606:2 2562:1 516:1 2570:2 524:2 6669:1 2576:1 2577:1 4626:1 6678:1 2584:1 6916:2 538:1 7600:1 547:2 549:2 553:9 555:1 2337:1 4655:1 567:1 5679:1 570:2 6722:2 579:2 6727:2 4793:1 586:1 590:4 2643:15 4694:14 4696:6 4698:1 603:3 4700:1 6749:1 6294:1 4704:1 613:1 4710:2 2833:1 6247:1 1469:1 6769:1 6770:1 629:1 4727:1 2682:4 640:1 642:1 6793:1 2703:1 659:6 772:1 664:1 2714:1 1135:4 3525:1 4768:2 674:1 678:1 4783:1 7624:2 690:1 115:1 1481:1 697:4 6843:1 2748:1 2753:2 6262:2 6854:1 4807:1 6856:2 2763:2 6863:1 2770:1 5923:3 6869:1 4824:2 4834:2 1489:1 2793:4 4844:2 4848:2 2801:1 755:1 2807:1 763:1 2815:2 1152:1 2818:2 2820:2 7638:1 778:1 6923:1 2831:4 6929:1 4882:1 4887:2 4888:16 6940:6 798:2 6950:2 4904:2 809:1 4907:1 4909:4 2870:1 4919:3 4922:2 2879:6 4930:1 4932:5 2892:1 842:1 6988:1 846:1 4943:1 6999:3 4952:1 864:1 4966:5 1853:2 2929:1 7026:2 5267:1 4984:1 4987:1 894:1 6440:1 7042:1 7045:1 4998:3 2953:2 7050:1 2955:3 7053:2 5014:1 836:1 5018:1 3443:2 924:4 7071:8 7072:1 930:1 936:3 5033:3 5036:1 942:2 2991:1 5047:1 7096:1 7099:2 3005:1 3006:3 3008:1 962:3 963:1 3013:1 967:2 5065:3 2419:1 5068:1 5070:1 976:1 977:1 7125:1 3031:1 7130:24 3039:3 7137:2 5090:1 5091:2 996:17 997:3 3047:2 7147:1 7149:1 5105:1 3060:1 3062:13 7159:1 5112:1 3066:4 5631:1 1022:1 1023:1 7171:1 5126:4 1032:1 5131:4 3087:1 2904:1 3090:1 7187:3 5147:1 3100:1 7200:2 7201:4 1058:2 7203:5 5156:2 7207:2 1065:6 5162:3 3116:6 5165:1 7214:1 3119:1 7222:1 5180:1 3133:1 1086:2 5183:15 7233:1 5188:2 7239:4 5192:1 1097:1 5194:2 405:1 4621:1 5200:1 3153:1 855:1 7252:2 1112:1 5211:7 7675:2 7264:1 5218:2 2235:1 5220:1 3173:1 1129:2 1130:5 3181:1 1134:1 7279:1 3184:1 3186:1 1139:1 191:1 3197:1 5248:2 5249:1 993:1 2582:1 1160:2 1165:1 7315:2 3223:1 7321:1 3229:2 4293:1 2631:1 7334:7 3239:3 7338:3 3243:1 5293:2 7344:1 7348:1 6345:3 1226:1 1216:2 3041:1 2361:1 3445:1 3273:1 7370:2 3277:1 3280:4 7378:1 7381:1 3287:4 3288:1 3295:2 6520:1 5348:1 5349:5 7398:1 3303:1 5354:1 5357:1 5358:2 7408:1 5365:2 4991:1 5372:2 7421:1 5374:8 5376:1 1921:1 7434:1 3342:1 1295:1 1296:1 3349:3 6361:1 1306:2 1583:1 5409:3 6113:1 2950:1 3975:1 5420:11 7469:1 1928:1 3381:2 1334:1 5001:5 5434:1 7391:2 1341:1 7487:1 1345:2 7491:1 5449:1 1355:1 2957:1 7505:2 5458:6 3114:1 5460:2 3641:2 7512:1 5466:1 5470:1 5350:1 7526:1 7529:1 7531:1 1388:2 5488:1 1395:3 7541:2 7546:1 1258:1 1407:1 3456:2 7555:2 7557:1 7558:1 5511:2 7560:1 7563:1 4674:1 1424:2 7576:4 3483:3 1437:2 5535:3 7584:1 5539:1 1449:1 5231:1 5548:1 5549:5 3503:1 5552:1 1458:1 5556:1 7611:1 3517:2 3317:3 5570:2 1477:6 5576:2 5577:1 3530:1 3531:1 1485:1 5585:1 7210:1 1492:1 5590:2 5591:1 3544:1 118:1 1502:1 3551:1 3558:3 1513:1 5612:1 3565:2 6397:1 5616:1 4691:2 5622:7 7671:1 3577:1 5626:1 6393:1 1532:2 5629:1 3583:2 7683:2 3590:3 7689:1 5644:1 5650:12 7699:1 5654:3 5655:1 3616:1 1569:1 1572:1 4485:3 5678:4 3631:16 5683:1 5686:1 5687:1 5688:2 5689:5 3646:4 3648:3 1608:15 951:1 5718:2 1625:2 3692:2 274:1 1646:4 3695:1 5751:1 5762:2 3727:3 3737:1 1690:3 5787:1 5794:1 3747:3 5799:4 5805:1 5808:5 3763:4 1716:2 287:1 1725:1 5825:1 7559:1 7457:4 3785:2 5834:1 1746:1 3795:1 1751:15 5859:1 1764:6 5863:1 4392:1 1789:1 5896:1 3860:3 1813:5 5912:1 1822:5 1826:1 3875:6 1828:1 3879:3 3880:1 353:2 3885:6 5934:1 3890:1 6451:2 5946:8 5947:1 3901:3 2653:3 3905:2 5955:2 3908:2 1861:1 1862:1 5959:1 1494:1 5431:1 7139:4 3925:4 5974:1 5975:1 3931:1 1884:3 881:1 1888:1 4411:1 3944:2 3948:1 3949:1 3951:2 3956:5 1910:1 3961:1 6010:1 1918:2 6016:1 320:4 5441:1 3976:1 6027:2 3985:1 1947:1 6045:3 4001:1 6811:1 4009:4 1965:1 1966:1 1967:1 328:1 6131:1 4085:2 1985:1 6083:1 4036:1 4039:1 6135:1 1996:3 6093:1 1999:1 1016:1 4054:5 4055:1 4060:1 2016:2 4432:1 4073:1 2028:5 2035:1 6133:1 2039:5 4436:1"); float score = VowpalWabbitInterface.Learn(vw, example); for (int i = 0; i < 10; i++) { float topicPrediction = VowpalWabbitInterface.GetTopicPrediction(example, (IntPtr)i); Console.Write("{0} ", topicPrediction); } Console.Write("\n"); VowpalWabbitInterface.FinishExample(vw, example); } private static void RunVWParse_and_VWLearn() { // parse and cache IntPtr vw0 = VowpalWabbitInterface.Initialize(@"-d 0002.dat -c"); VowpalWabbitInterface.StartParser(vw0, false); long instanceCount = 0; VWInstanceEx[] vwInstanceExs = new VWInstanceEx[781266]; Stopwatch s = Stopwatch.StartNew(); while (instanceCount < 781266) { IntPtr example = VowpalWabbitInterface.GetExample(vw0); if (IntPtr.Zero == example) break; vwInstanceExs[instanceCount] = new VWInstanceEx(vw0, example); VowpalWabbitInterface.FinishExample(vw0, example); instanceCount++; } VowpalWabbitInterface.EndParser(vw0); VowpalWabbitInterface.Finish(vw0); // learn instanceCount = 0; IntPtr vw = VowpalWabbitInterface.Initialize(@"--quiet --random_seed 276518665 -f save_file.reg --readable_model reable.reg"); foreach (VWInstanceEx vwInstanceEx in vwInstanceExs) { VowpalWabbitInterface.FEATURE_SPACE[] featureSpace = new VowpalWabbitInterface.FEATURE_SPACE[vwInstanceEx.featureSpace.Length]; GCHandle[] pinnedsFeatures = new GCHandle[vwInstanceEx.featureSpace.Length]; for (int i = 0; i < vwInstanceEx.featureSpace.Length; i++) { pinnedsFeatures[i] = GCHandle.Alloc(vwInstanceEx.featureSpace[i].features, GCHandleType.Pinned); featureSpace[i].features = pinnedsFeatures[i].AddrOfPinnedObject(); } GCHandle pinnedFeatureSpace = GCHandle.Alloc(featureSpace, GCHandleType.Pinned); IntPtr featureSpacePtr = pinnedFeatureSpace.AddrOfPinnedObject(); IntPtr importedExample = VowpalWabbitInterface.ImportExample(vw, featureSpacePtr, (IntPtr)vwInstanceEx.featureSpace.Length); VowpalWabbitInterface.Learn(vw, importedExample); VowpalWabbitInterface.FinishExample(vw, importedExample); for (int i = 0; i < vwInstanceEx.featureSpace.Length; i++) { pinnedsFeatures[i].Free(); } pinnedFeatureSpace.Free(); importedExample = IntPtr.Zero; instanceCount++; } VowpalWabbitInterface.Finish(vw); Debug.WriteLine("Elapsed Time: {0} ms", s.ElapsedMilliseconds); Console.WriteLine("Elapsed Time: {0} ms", s.ElapsedMilliseconds); } } } vowpal-wabbit-8.5.0.dfsg1/cs/test/Properties/000077500000000000000000000000001323713302100210205ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/test/Properties/AssemblyInfo.cs000066400000000000000000000025661323713302100237530ustar00rootroot00000000000000using System.Reflection; using System.Runtime.CompilerServices; using System.Runtime.InteropServices; // General Information about an assembly is controlled through the following // set of attributes. Change these attribute values to modify the information // associated with an assembly. [assembly: AssemblyTitle("cs_test")] [assembly: AssemblyDescription("")] [assembly: AssemblyConfiguration("")] [assembly: AssemblyCompany("MSIT")] [assembly: AssemblyProduct("cs_test")] [assembly: AssemblyCopyright("Copyright © MSIT 2012")] [assembly: AssemblyTrademark("")] [assembly: AssemblyCulture("")] // Setting ComVisible to false makes the types in this assembly not visible // to COM components. If you need to access a type in this assembly from // COM, set the ComVisible attribute to true on that type. [assembly: ComVisible(false)] // The following GUID is for the ID of the typelib if this project is exposed to COM [assembly: Guid("74451adb-817c-45fa-af74-71fd22936907")] // Version information for an assembly consists of the following four values: // // Major Version // Minor Version // Build Number // Revision // // You can specify all the values or you can default the Build and Revision Numbers // by using the '*' as shown below: // [assembly: AssemblyVersion("1.0.*")] [assembly: AssemblyVersion("1.0.0.0")] [assembly: AssemblyFileVersion("1.0.0.0")] vowpal-wabbit-8.5.0.dfsg1/cs/test/VowpalWabbitInterface.cs000066400000000000000000000145401323713302100234410ustar00rootroot00000000000000using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Runtime.InteropServices; namespace VW { using SizeT = IntPtr; using VwHandle = IntPtr; using VwFeatureSpace = IntPtr; using VwExample = IntPtr; using VwFeature = IntPtr; using BytePtr = IntPtr; public sealed class VowpalWabbitInterface { private const string LIBVW = "libvw.dll"; [StructLayout(LayoutKind.Sequential)] public struct FEATURE_SPACE { public byte name; public IntPtr features; // points to a FEATURE[] public int len; } [StructLayout(LayoutKind.Sequential)] public struct FEATURE { public float x; public uint weight_index; } [DllImport(LIBVW, EntryPoint = "VW_Initialize")] public static extern VwHandle Initialize([MarshalAs(UnmanagedType.LPWStr)]string arguments); [DllImport(LIBVW, EntryPoint = "VW_Finish")] public static extern void Finish(VwHandle vw); [DllImport(LIBVW, EntryPoint = "VW_ImportExample")] // features points to a FEATURE_SPACE[] public static extern VwExample ImportExample(VwHandle vw, VwFeatureSpace features, SizeT length); [DllImport(LIBVW, EntryPoint = "VW_ExportExample")] public static extern VwFeatureSpace ExportExample(VwHandle vw, VwExample example, ref SizeT length); [DllImport(LIBVW, EntryPoint = "VW_ReleaseFeatureSpace")] public static extern void ReleaseFeatureSpace(VwFeatureSpace fs, SizeT length); [DllImport(LIBVW, EntryPoint = "VW_ReadExample")] public static extern VwExample ReadExample(VwHandle vw, [MarshalAs(UnmanagedType.LPWStr)]string exampleString); // Have to marshal bools, C++ considers them 4 byte quantities, and C# considers them 1 byte. [DllImport(LIBVW, EntryPoint = "VW_StartParser")] public static extern void StartParser(VwHandle vw, [MarshalAs(UnmanagedType.Bool)]bool do_init); [DllImport(LIBVW, EntryPoint = "VW_EndParser")] public static extern void EndParser(VwHandle vw); [DllImport(LIBVW, EntryPoint = "VW_GetExample")] public static extern VwExample GetExample(VwHandle parser); [DllImport(LIBVW, EntryPoint = "VW_FinishExample")] public static extern void FinishExample(VwHandle vw, VwExample example); [DllImport(LIBVW, EntryPoint = "VW_GetTopicPrediction")] public static extern float GetTopicPrediction(VwExample example, SizeT i); [DllImport(LIBVW, EntryPoint = "VW_GetLabel")] public static extern float GetLabel(VwExample example); [DllImport(LIBVW, EntryPoint = "VW_GetImportance")] public static extern float GetImportance(VwExample example); [DllImport(LIBVW, EntryPoint = "VW_GetInitial")] public static extern float GetInitial(VwExample example); [DllImport(LIBVW, EntryPoint = "VW_GetMultilabelPredictions")] public static extern IntPtr GetMultilabelPredictions(VwHandle vw, VwExample example, ref SizeT length); [DllImport(LIBVW, EntryPoint = "VW_GetPrediction")] public static extern float GetPrediction(VwExample example); [DllImport(LIBVW, EntryPoint = "VW_GetTagLength")] public static extern SizeT GetTagLength(VwExample example); // Saying this returned a byte was inappropriate, because you were returning // actually a pointer to a seqeunce of bytes. (Not sure what the interpretation // of this should be, utf8 or something?) [DllImport(LIBVW, EntryPoint = "VW_GetTag")] public static extern BytePtr GetTag(VwExample example); [DllImport(LIBVW, EntryPoint = "VW_GetFeatureNumber")] public static extern SizeT GetFeatureNumber(VwExample example); // Same note regarding ref int vs size_t* [DllImport(LIBVW, EntryPoint = "VW_GetFeatures")] public static extern VwFeature GetFeatures(VwHandle vw, VwExample example, ref SizeT length); [DllImport(LIBVW, EntryPoint = "VW_ReturnFeatures")] public static extern void ReturnFeatures(VwExample features); [DllImport(LIBVW, EntryPoint = "VW_HashSpace")] public static extern uint HashSpace(VwHandle vw, [MarshalAs(UnmanagedType.LPWStr)]string s); [DllImport(LIBVW, EntryPoint = "VW_HashSpaceStatic")] public static extern uint HashSpace([MarshalAs(UnmanagedType.LPWStr)]string s, [MarshalAs(UnmanagedType.LPWStr)]string h = "strings"); // The DLL defines the last argument "u" as being an "unsigned long". // In C++ under current circumstances, both ints and longs are four byte integers. // If you wanted an eight byte integer you should use "long long" (or probably // more appropriately in this circumstance size_t). // In C#, "int" is four bytes, "long" is eight bytes. [DllImport(LIBVW, EntryPoint = "VW_HashFeature")] public static extern uint HashFeature(VwHandle vw, [MarshalAs(UnmanagedType.LPWStr)]string s, uint u); [DllImport(LIBVW, EntryPoint = "VW_HashFeatureStatic")] public static extern uint HashFeature([MarshalAs(UnmanagedType.LPWStr)]string s, uint u, [MarshalAs(UnmanagedType.LPWStr)]string h = "strings", uint numBits = 18); [DllImport(LIBVW, EntryPoint = "VW_Learn")] public static extern float Learn(VwHandle vw, VwExample example); [DllImport(LIBVW, EntryPoint = "VW_Predict")] public static extern float Predict(VwHandle vw, VwExample example); [DllImport(LIBVW, EntryPoint = "VW_AddLabel")] public static extern void AddLabel(VwExample example, float label = float.MaxValue, float weight = 1, float initial = 0); [DllImport(LIBVW, EntryPoint = "VW_Get_Weight")] public static extern float Get_Weight(VwHandle vw, SizeT index, SizeT offset); [DllImport(LIBVW, EntryPoint = "VW_Set_Weight")] public static extern void Set_Weight(VwHandle vw, SizeT index, SizeT offset, float value); [DllImport(LIBVW, EntryPoint = "VW_Num_Weights")] public static extern SizeT Num_Weights(VwHandle vw); [DllImport(LIBVW, EntryPoint = "VW_Get_Stride")] public static extern SizeT Get_Stride(VwHandle vw); [DllImport(LIBVW, EntryPoint = "VW_SaveModel")] public static extern void SaveModel(VwHandle vw); } } vowpal-wabbit-8.5.0.dfsg1/cs/test/app.config000066400000000000000000000037101323713302100206340ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/test/cs_test.csproj000066400000000000000000000324721323713302100215620ustar00rootroot00000000000000 Debug x86 8.0.30703 2.0 {D5E462FC-3DD6-4B65-A9E9-DA0B0B11D254} Exe Properties cs_test cs_test v4.6 512 false publish\ true Disk false Foreground 7 Days false false true 0 1.0.0.%2a false true ..\..\vowpalwabbit\ x86 true full false bin\Debug\ DEBUG;TRACE prompt 4 false x86 pdbonly true bin\Release\ TRACE prompt 4 false true $(SolutionDir)\x64\Debug\ DEBUG;TRACE full x64 bin\Debug\cs_test.exe.CodeAnalysisLog.xml true GlobalSuppressions.cs prompt MinimumRecommendedRules.ruleset ;C:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Static Analysis Tools\\Rule Sets false ;C:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Static Analysis Tools\FxCop\\Rules false false false $(SolutionDir)\x64\Release\ TRACE true pdbonly x64 bin\Release\cs_test.exe.CodeAnalysisLog.xml true GlobalSuppressions.cs prompt MinimumRecommendedRules.ruleset ;C:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Static Analysis Tools\\Rule Sets false ;C:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Static Analysis Tools\FxCop\\Rules false false false true bin\Debug\ DEBUG;TRACE full x86 bin\Debug\cs_test.exe.CodeAnalysisLog.xml true GlobalSuppressions.cs prompt MinimumRecommendedRules.ruleset ;C:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Static Analysis Tools\\Rule Sets;C:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Static Analysis Tools\\Rule Sets false ;C:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Static Analysis Tools\FxCop\\Rules;C:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Static Analysis Tools\FxCop\\Rules false false bin\Release\ TRACE true pdbonly x86 bin\Release\cs_test.exe.CodeAnalysisLog.xml true GlobalSuppressions.cs prompt MinimumRecommendedRules.ruleset ;C:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Static Analysis Tools\\Rule Sets;C:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Static Analysis Tools\\Rule Sets ;C:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Static Analysis Tools\FxCop\\Rules;C:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Static Analysis Tools\FxCop\\Rules false true bin\x86\DebugLeakCheck\ DEBUG;TRACE full x86 prompt MinimumRecommendedRules.ruleset true bin\x64\DebugLeakCheck\ DEBUG;TRACE full x64 false prompt MinimumRecommendedRules.ruleset false false true bin\DebugLeakCheck\ DEBUG;TRACE full x86 prompt MinimumRecommendedRules.ruleset false false {e4e962ae-7056-4eb0-a8c5-8dc824a4b068} cs {9e27fa94-ab34-4736-8427-fb7a2ba90d52} cs_json {08636f79-5577-4af2-8eed-ec8a5bc14ac4} cs_parallel {007b7de0-1d9b-498d-acfc-e9d33058f22e} cs_unittest {85e55ae0-3784-4968-9271-c81af560e1c1} vw_clr {e621e022-c1f8-433f-905a-ab9a3de072b7} vw_common False Microsoft .NET Framework 4 Client Profile %28x86 and x64%29 true False .NET Framework 3.5 SP1 Client Profile false False .NET Framework 3.5 SP1 false False Windows Installer 4.5 true Designer xcopy /y/d $(SolutionDir)\dll\$(PlatformName)\$(ConfigurationName)\libvw.* $(TargetDir) This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. vowpal-wabbit-8.5.0.dfsg1/cs/testcommon/000077500000000000000000000000001323713302100200755ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/testcommon/ITestRunner.cs000066400000000000000000000007051323713302100226500ustar00rootroot00000000000000using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; namespace cs_testcommon { public interface ITestRunner { /// /// /// /// /// /// Null if ok, otherwise the test message string Run(string type, string method); } } vowpal-wabbit-8.5.0.dfsg1/cs/testcommon/Properties/000077500000000000000000000000001323713302100222315ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/testcommon/Properties/AssemblyInfo.cs000066400000000000000000000025721323713302100251610ustar00rootroot00000000000000using System.Reflection; using System.Runtime.CompilerServices; using System.Runtime.InteropServices; // General Information about an assembly is controlled through the following // set of attributes. Change these attribute values to modify the information // associated with an assembly. [assembly: AssemblyTitle("cs_testcommon")] [assembly: AssemblyDescription("")] [assembly: AssemblyConfiguration("")] [assembly: AssemblyCompany("")] [assembly: AssemblyProduct("cs_testcommon")] [assembly: AssemblyCopyright("Copyright © 2015")] [assembly: AssemblyTrademark("")] [assembly: AssemblyCulture("")] // Setting ComVisible to false makes the types in this assembly not visible // to COM components. If you need to access a type in this assembly from // COM, set the ComVisible attribute to true on that type. [assembly: ComVisible(false)] // The following GUID is for the ID of the typelib if this project is exposed to COM [assembly: Guid("32fb833b-e31d-4326-90a8-21bd8ca70c12")] // Version information for an assembly consists of the following four values: // // Major Version // Minor Version // Build Number // Revision // // You can specify all the values or you can default the Build and Revision Numbers // by using the '*' as shown below: // [assembly: AssemblyVersion("1.0.*")] [assembly: AssemblyVersion("1.0.0.0")] [assembly: AssemblyFileVersion("1.0.0.0")] vowpal-wabbit-8.5.0.dfsg1/cs/testcommon/cs_testcommon.csproj000066400000000000000000000063301323713302100241760ustar00rootroot00000000000000 Debug AnyCPU {6A9CBEAB-427F-4D8D-9559-B76B42B0895C} Library Properties cs_testcommon cs_testcommon v4.5.2 512 true $(SolutionDir)\x64\Debug\ DEBUG;TRACE full x64 prompt MinimumRecommendedRules.ruleset true $(SolutionDir)\x64\Release\ TRACE true pdbonly x64 prompt MinimumRecommendedRules.ruleset true ..\vw_key.snk true $(SolutionDir)\x64\DebugLeakCheck\ DEBUG;TRACE true full x64 prompt MinimumRecommendedRules.ruleset vowpal-wabbit-8.5.0.dfsg1/cs/testhelp/000077500000000000000000000000001323713302100175355ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/testhelp/App.config000066400000000000000000000037101323713302100214450ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/testhelp/DiscoverUnitTests.cs000066400000000000000000000045621323713302100235340ustar00rootroot00000000000000using cs_unittest; using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Reflection; using System.Text; using System.Threading.Tasks; using Microsoft.VisualStudio.TestTools.UnitTesting; namespace cs_testhelp { internal static class DiscoverUnitTests { internal static void Discover(string vwRoot) { // find all methods that are // - not part of types annotated by [TestClass] (as they'll already be discovered by unit test framework) // - contain "test" in method name var methodByType = from type in typeof(TestRunner).Assembly.GetTypes() where type.GetCustomAttribute() != null from method in type.GetMethods(BindingFlags.Instance | BindingFlags.Public) where method.Name.ToLowerInvariant().Contains("test") && method.GetParameters().Count() == 0 && method.GetCustomAttribute() != null && method.GetCustomAttribute() == null group method by method.DeclaringType into g select g; var outputFile = vwRoot + @"\..\cs\leaktest\TestWrapped.cs"; using (var cs = new StreamWriter(outputFile)) { cs.WriteLine(@" using Microsoft.VisualStudio.TestTools.UnitTesting; using cs_leaktest; namespace cs_unittest { [TestClass] public class TestWrapped : TestWrappedBase {"); foreach (var g in methodByType) { foreach (var method in g) { var name = method.Name; var categoryAttr = method.GetCustomAttribute(); if (categoryAttr != null) { cs.WriteLine(@" [TestCategory({0})]", string.Join(",", categoryAttr.TestCategories.Select(c => string.Format("\"{0}\"", c)))); } cs.WriteLine(@" [TestMethod] public void {1}() {{ Run(""{0}"", ""{1}""); }}", g.Key.FullName, name); } } cs.WriteLine(@" } } "); } } } } vowpal-wabbit-8.5.0.dfsg1/cs/testhelp/Program.cs000066400000000000000000000006061323713302100214750ustar00rootroot00000000000000using System; using System.Collections.Generic; using System.Globalization; using System.IO; using System.Linq; using System.Text; using System.Text.RegularExpressions; using VW; namespace cs_testhelp { class Program { static void Main(string[] mainArgs) { var vwRoot = mainArgs[0]; DiscoverUnitTests.Discover(vwRoot); } } } vowpal-wabbit-8.5.0.dfsg1/cs/testhelp/Properties/000077500000000000000000000000001323713302100216715ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/testhelp/Properties/AssemblyInfo.cs000066400000000000000000000025661323713302100246240ustar00rootroot00000000000000using System.Reflection; using System.Runtime.CompilerServices; using System.Runtime.InteropServices; // General Information about an assembly is controlled through the following // set of attributes. Change these attribute values to modify the information // associated with an assembly. [assembly: AssemblyTitle("cs_testhelp")] [assembly: AssemblyDescription("")] [assembly: AssemblyConfiguration("")] [assembly: AssemblyCompany("")] [assembly: AssemblyProduct("cs_testhelp")] [assembly: AssemblyCopyright("Copyright © 2015")] [assembly: AssemblyTrademark("")] [assembly: AssemblyCulture("")] // Setting ComVisible to false makes the types in this assembly not visible // to COM components. If you need to access a type in this assembly from // COM, set the ComVisible attribute to true on that type. [assembly: ComVisible(false)] // The following GUID is for the ID of the typelib if this project is exposed to COM [assembly: Guid("f02375f2-79db-4800-8982-7c8f25ef4624")] // Version information for an assembly consists of the following four values: // // Major Version // Minor Version // Build Number // Revision // // You can specify all the values or you can default the Build and Revision Numbers // by using the '*' as shown below: // [assembly: AssemblyVersion("1.0.*")] [assembly: AssemblyVersion("1.0.0.0")] [assembly: AssemblyFileVersion("1.0.0.0")] vowpal-wabbit-8.5.0.dfsg1/cs/testhelp/cs_testhelp.csproj000066400000000000000000000146431323713302100233040ustar00rootroot00000000000000 Debug AnyCPU {B7122114-828E-4D86-B79E-4037C74C5F69} Exe Properties cs_testhelp cs_testhelp v4.6 512 ..\..\vowpalwabbit true $(SolutionDir)\x64\Debug\ DEBUG;TRACE full x64 prompt MinimumRecommendedRules.ruleset true $(SolutionDir)\x64\Release\ TRACE true pdbonly x64 prompt MinimumRecommendedRules.ruleset true true ..\..\vowpalwabbit\x64\DebugLeakCheck\ DEBUG;TRACE full x64 prompt MinimumRecommendedRules.ruleset true ..\..\vowpalwabbit\packages\Antlr4.Runtime.4.6.4\lib\net45\Antlr4.Runtime.dll True {6a9cbeab-427f-4d8d-9559-b76b42b0895c} cs_testcommon {007b7de0-1d9b-498d-acfc-e9d33058f22e} cs_unittest {85e55ae0-3784-4968-9271-c81af560e1c1} vw_clr This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. vowpal-wabbit-8.5.0.dfsg1/cs/testhelp/packages.config000066400000000000000000000005261323713302100225050ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/unittest/000077500000000000000000000000001323713302100175645ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/unittest/App.config000066400000000000000000000127211323713302100214760ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/unittest/ApplicationInsights.config000066400000000000000000000027731323713302100247400ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/unittest/BaseData.cs000066400000000000000000000001541323713302100215570ustar00rootroot00000000000000namespace cs_unittest { public class BaseData { public string Line { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/CbAdfData.cs000066400000000000000000000027071323713302100216520ustar00rootroot00000000000000using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using VW.Labels; using VW.Serializer.Attributes; namespace cs_unittest.cbadf { public class Generator { private static Random rand = new Random(123); public static Tuple, ContextualBanditLabel> GenerateShared(int numActions) { return Tuple.Create( new CbAdfShared { Num = rand.Next(100), Vector = Enumerable.Range(1, 500).Select(_ => (float)rand.NextDouble()).ToArray() }, Enumerable.Range(1, numActions).Select(_ => new CbAdfAction { Vector = Enumerable.Range(1, 500).Select(__ => (float)rand.NextDouble()).ToArray() }).ToList(), new ContextualBanditLabel { Action = (uint)rand.Next(numActions), Cost = rand.Next(1), Probability = (float)rand.NextDouble() }); } } public class CbAdfShared { [Feature] public int Num { get; set; } [Feature(FeatureGroup = 'x', AddAnchor = true)] public float[] Vector { get; set; } } public class CbAdfAction { [Feature(FeatureGroup = 'y', AddAnchor = true)] public float[] Vector { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/Properties/000077500000000000000000000000001323713302100217205ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/unittest/Properties/AssemblyInfo.cs000066400000000000000000000026371323713302100246520ustar00rootroot00000000000000using System; using System.Reflection; using System.Runtime.CompilerServices; using System.Runtime.InteropServices; // General Information about an assembly is controlled through the following // set of attributes. Change these attribute values to modify the information // associated with an assembly. [assembly: AssemblyTitle("cs_unittest")] [assembly: AssemblyDescription("")] [assembly: AssemblyConfiguration("")] [assembly: AssemblyCompany("")] [assembly: AssemblyProduct("cs_unittest")] [assembly: AssemblyCopyright("Copyright © 2015")] [assembly: AssemblyTrademark("")] [assembly: AssemblyCulture("")] // Setting ComVisible to false makes the types in this assembly not visible // to COM components. If you need to access a type in this assembly from // COM, set the ComVisible attribute to true on that type. [assembly: ComVisible(false)] // The following GUID is for the ID of the typelib if this project is exposed to COM [assembly: Guid("424e177f-656d-4926-879f-ffac8a250103")] // Version information for an assembly consists of the following four values: // // Major Version // Minor Version // Build Number // Revision // // You can specify all the values or you can default the Build and Revision Numbers // by using the '*' as shown below: // [assembly: AssemblyVersion("1.0.*")] [assembly: AssemblyVersion("1.0.0.0")] [assembly: AssemblyFileVersion("1.0.0.0")] [assembly: CLSCompliant(false)] vowpal-wabbit-8.5.0.dfsg1/cs/unittest/RunTests.tt000066400000000000000000000114001323713302100217200ustar00rootroot00000000000000<#@ template debug="false" hostspecific="True" language="C#" #> <#@ assembly name="System.Core" #> <#@ import namespace="System.Linq" #> <#@ import namespace="System.IO" #> <#@ import namespace="System.Globalization" #> <#@ import namespace="System.Text" #> <#@ import namespace="System.Text.RegularExpressions" #> <#@ import namespace="System.Collections.Generic" #> <#@ output extension=".cs" #> <# var testRoot = Path.Combine(Path.GetDirectoryName(Host.TemplateFile), @"..\..\test"); var lines = File.ReadAllLines(Path.Combine(testRoot, "RunTests")) .SkipWhile(l => l != "__DATA__") .ToList(); var skipList = new[] { 13, 32, 39, 40, 41, 59, 60, 61, 66, 68, 90, 25, 26, // crash 92, 95, 96, 98, 91, 99, 118, 119, 120, 14, 16, 17, 31, 33, 34,53, 101, 102, 103, 105, 106, 111, 112, // float delta 71, // --examples to test parser 143, 144, 146, 158, // native json parsing 149, 152, 156 // bash script }; var outputModels = new Dictionary(); var testcases = new Dictionary(); testcase = new TestCase(); Match match; for (var i = 0; i < lines.Count; i++) { var line = lines[i]; if (line.Trim().Length == 0) { if (skipTest) { Reset(); continue; } // resolve dependencies if (!string.IsNullOrEmpty(testcase.FinalRegressor)) outputModels[testcase.FinalRegressor] = testcase; if (!string.IsNullOrEmpty(testcase.InitialRegressor)) { TestCase dep; if (outputModels.TryGetValue(testcase.InitialRegressor, out dep)) testcase.Dependency = dep; else throw new Exception("Missing dependency: '" + testcase.InitialRegressor + "' for test case " + testcase.Id); } // if (testcase.Id == 31) // testcase.Dependency = outputModels["models/mask.model"]; if (testcases.ContainsKey(testcase.Id)) throw new Exception("Testcase IDs must be unique: duplicate # Test " + testcase.Id + " in line " + i); testcases.Add(testcase.Id, testcase); Reset(); } else if ((match = Regex.Match(line, @"^#\s*Test\s+(?\d+):?(?.*)?$")).Success) { var nr = int.Parse(match.Groups["nr"].Value); testcase.Id = nr; testcase.Comment = match.Groups["comment"].Value.Replace("\"", "\"\""); } else if ((match = Regex.Match(line, @"^\{VW\} (?.*)$")).Success) { var args = match.Groups["args"].Value; while (args.EndsWith("\\")) { args = args.Substring(0, args.Length - 1); args = args.Trim() + " " + lines[++i].Trim(); } testcase.Arguments = args; testcase.InputData = MatchArgument(args, "-d"); testcase.InitialRegressor = MatchArgument(args, "-i"); testcase.FinalRegressor = MatchArgument(args, "-f"); } else if (line.EndsWith(".stderr")) { testcase.Stderr = line.Trim(); } else if (line.EndsWith(".predict")) { testcase.Predict = line.Trim(); } else if (line.StartsWith("#") && line.Contains("SkipC#")) { skipTest = true; } } #> using Microsoft.VisualStudio.TestTools.UnitTesting; using System.IO; using System.IO.Compression; using VW; namespace cs_unittest { [TestClass] public partial class RunTests : TestBase { <# foreach (var mainTestcase in testcases.Values) { if (mainTestcase.Id == 0) continue; #> [TestMethod] [Description(@"<#=(mainTestcase.Comment ?? "").Trim()#>")] <# if (skipList.Contains(mainTestcase.Id)) { #> [Ignore] <# } #> [TestCategory("Vowpal Wabbit/Command Line")] public void CommandLine_Test<#=mainTestcase.Id#>() { <# foreach (var tc in mainTestcase.InDependencyOrder()) { #> RunTestsHelper.ExecuteTest( <#=tc.Id#>, "<#=tc.Arguments#>", "<#=tc.InputData#>", "<#=tc.Stderr#>", "<#=tc.Predict#>"); <# } #> } <# } #> } } <#+ class TestCase { public int Id; public string Arguments = ""; public string InitialRegressor; public string FinalRegressor; public string InputData = ""; public string Stderr = ""; public string Predict = ""; public string Comment; public TestCase Dependency; public List InDependencyOrder() { var tests = new List(); var dep = this; while (dep != null) { tests.Add(dep); dep = dep.Dependency; } tests.Reverse(); return tests; } } TestCase testcase; bool skipTest = false; private void Reset() { testcase = new TestCase(); skipTest = false; } private string MatchArgument(string args, string option) { var match = Regex.Match(args, Regex.Escape(option) + @"\s+(?\S+)"); return match.Success ? match.Groups["value"].Value : ""; } #>vowpal-wabbit-8.5.0.dfsg1/cs/unittest/RunTestsHelper.cs000066400000000000000000000125241323713302100230460ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections.Generic; using System.Globalization; using System.IO; using System.IO.Compression; using System.Linq; using System.Text; using System.Threading.Tasks; using VW; namespace cs_unittest { public static class RunTestsHelper { private static StreamReader Open(string input) { if (input.EndsWith(".gz")) return new StreamReader(new GZipStream(new FileStream(input, FileMode.Open), CompressionMode.Decompress)); else return new StreamReader(input); } private static bool IsMultilineData(string input) { using (var streamReader = Open(input)) { string dataLine; while ((dataLine = streamReader.ReadLine()) != null) { if (string.IsNullOrWhiteSpace(dataLine)) { return true; } } } return false; } public static void ExecuteTest(int testCaseNr, string args, string input, string stderr, string predictFile) { using (var vw = new VowpalWabbit(args)) { var multiline = IsMultilineData(input); using (var streamReader = Open(input)) { if (multiline) { var lines = new List(); string dataLine; while ((dataLine = streamReader.ReadLine()) != null) { if (string.IsNullOrWhiteSpace(dataLine)) { if (lines.Count > 0) { if (args.Contains("-t")) // test only vw.Predict(lines); else vw.Learn(lines); } lines.Clear(); continue; } lines.Add(dataLine); } } else { int lineNr = 0; string[] predictions = null; if (File.Exists(predictFile)) predictions = File.ReadAllLines(predictFile); string dataLine; while ((dataLine = streamReader.ReadLine()) != null) { if (!string.IsNullOrWhiteSpace(predictFile) && File.Exists(predictFile)) { object actualValue; if (args.Contains("-t")) // test only actualValue = vw.Predict(dataLine, VowpalWabbitPredictionType.Dynamic); else actualValue = vw.Learn(dataLine, VowpalWabbitPredictionType.Dynamic); if (predictions != null) { // validate predictions var actualFloat = actualValue as float?; if (actualFloat != null) { var expectedPrediction = float.Parse(predictions[lineNr].Split(' ').First(), CultureInfo.InvariantCulture); VWTestHelper.FuzzyEqual(expectedPrediction, (float)actualFloat, 1e-4, "Prediction mismatch"); } var actualScalar = actualValue as VowpalWabbitScalar?; if (actualScalar != null) { var expectedPredictions = predictions[lineNr] .Split(' ') .Select(field => float.Parse(field, CultureInfo.InvariantCulture)) .ToArray(); Assert.AreEqual(2, expectedPredictions.Length); VWTestHelper.FuzzyEqual(expectedPredictions[0], actualScalar.Value.Value, 1e-4, "Prediction value mismatch"); VWTestHelper.FuzzyEqual(expectedPredictions[1], actualScalar.Value.Confidence, 1e-4, "Prediction confidence mismatch"); } } } else vw.Learn(dataLine); lineNr++; } } if (vw.Arguments.NumPasses > 1) vw.RunMultiPass(); else vw.EndOfPass(); if (!string.IsNullOrWhiteSpace(stderr) && File.Exists(stderr)) VWTestHelper.AssertEqual(stderr, vw.PerformanceStatistics); } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/Test1and2.cs000066400000000000000000000171121323713302100216620ustar00rootroot00000000000000using System; using System.Collections.Generic; using System.Globalization; using System.IO; using System.Linq; using cs_unittest; using Microsoft.VisualStudio.TestTools.UnitTesting; using VW; using VW.Labels; using VW.Serializer.Attributes; using System.Threading; using VW.Serializer; using cs_testcommon; namespace cs_test { [TestClass] public class Test1and2Class : TestBase { [TestMethod] [TestCategory("Vowpal Wabbit/Command line through marshalling")] public void Test1and2() { var references = File.ReadAllLines(@"pred-sets\ref\0001.predict").Select(l => float.Parse(l, CultureInfo.InvariantCulture)).ToArray(); var input = new List(); using (var vwStr = new VowpalWabbit(" -k -c test1and2.str --passes 8 -l 20 --power_t 1 --initial_t 128000 --ngram 3 --skips 1 --invariant --holdout_off")) using (var vw = new VowpalWabbit(new VowpalWabbitSettings(" -k -c test1and2 --passes 8 -l 20 --power_t 1 --initial_t 128000 --ngram 3 --skips 1 --invariant --holdout_off") { EnableExampleCaching = false })) using (var vwValidate = new VowpalWabbitExampleValidator("-l 20 --power_t 1 --initial_t 128000 --ngram 3 --skips 1 --invariant --holdout_off")) { var lineNr = 0; VWTestHelper.ParseInput( File.OpenRead(@"train-sets\0001.dat"), new MyListener(data => { input.Add(data); vwValidate.Validate(data.Line, data, data.Label); var expected = vwStr.Learn(data.Line, VowpalWabbitPredictionType.Dynamic); Assert.IsInstanceOfType(expected, typeof(float)); var actual = vw.Learn(data, data.Label, VowpalWabbitPredictionType.Scalar); Assert.AreEqual((float)expected, actual, 1e-6, "Learn output differs on line: " + lineNr); lineNr++; })); vwStr.RunMultiPass(); vw.Native.RunMultiPass(); vwStr.SaveModel("models/str0001.model"); vw.Native.SaveModel("models/0001.model"); VWTestHelper.AssertEqual(@"train-sets\ref\0001.stderr", vwStr.PerformanceStatistics); VWTestHelper.AssertEqual(@"train-sets\ref\0001.stderr", vw.Native.PerformanceStatistics); } Assert.AreEqual(input.Count, references.Length); using (var vwModel = new VowpalWabbitModel(new VowpalWabbitSettings("-k -t --invariant") { ModelStream = File.OpenRead("models/0001.model") })) using (var vwInMemoryShared1 = new VowpalWabbit(new VowpalWabbitSettings { Model = vwModel })) using (var vwInMemoryShared2 = new VowpalWabbit(new VowpalWabbitSettings { Model = vwModel })) using (var vwInMemory = new VowpalWabbit(new VowpalWabbitSettings("-k -t --invariant") { ModelStream = File.OpenRead("models/0001.model") })) using (var vwStr = new VowpalWabbit("-k -t -i models/str0001.model --invariant")) using (var vwNative = new VowpalWabbit("-k -t -i models/0001.model --invariant")) using (var vw = new VowpalWabbit("-k -t -i models/0001.model --invariant")) using (var vwModel2 = new VowpalWabbitModel("-k -t --invariant -i models/0001.model")) using (var vwInMemoryShared3 = new VowpalWabbit(new VowpalWabbitSettings { Model = vwModel2 })) { for (var i = 0; i < input.Count; i++) { var actualStr = vwStr.Predict(input[i].Line, VowpalWabbitPredictionType.Scalar); var actualNative = vwNative.Predict(input[i].Line, VowpalWabbitPredictionType.Scalar); var actualInMemory = vwInMemory.Predict(input[i].Line, VowpalWabbitPredictionType.Scalar); var actual = vw.Predict(input[i], VowpalWabbitPredictionType.Scalar, input[i].Label); var actualShared1 = vwInMemoryShared1.Predict(input[i].Line, VowpalWabbitPredictionType.Scalar); var actualShared2 = vwInMemoryShared2.Predict(input[i], VowpalWabbitPredictionType.Scalar, input[i].Label); var actualShared3 = vwInMemoryShared3.Predict(input[i], VowpalWabbitPredictionType.Scalar, input[i].Label); Assert.AreEqual(references[i], actualStr, 1e-5); Assert.AreEqual(references[i], actualNative, 1e-5); Assert.AreEqual(references[i], actualInMemory, 1e-5); Assert.AreEqual(references[i], actual, 1e-5); Assert.AreEqual(references[i], actualShared1, 1e-5); Assert.AreEqual(references[i], actualShared2, 1e-5); Assert.AreEqual(references[i], actualShared3, 1e-5); } // due to shared usage the counters don't match up //VWTestHelper.AssertEqual(@"test-sets\ref\0001.stderr", vwInMemoryShared2.Native.PerformanceStatistics); //VWTestHelper.AssertEqual(@"test-sets\ref\0001.stderr", vwInMemoryShared1.PerformanceStatistics); VWTestHelper.AssertEqual(@"test-sets\ref\0001.stderr", vwInMemory.PerformanceStatistics); VWTestHelper.AssertEqual(@"test-sets\ref\0001.stderr", vwStr.PerformanceStatistics); VWTestHelper.AssertEqual(@"test-sets\ref\0001.stderr", vw.Native.PerformanceStatistics); } } } // 1|features 13:.1 15:.2 const:25 // 1|abc 13:.1 15:.2 co:25 public class Test1 { [Feature(FeatureGroup = 'f', Namespace = "eatures", Name = "const", Order = 2)] public float Constant { get; set; } [Feature(FeatureGroup = 'f', Namespace = "eatures", Order = 1)] public IList> Features { get; set; } public string Line { get; set; } public ILabel Label { get; set;} } public class Rcv1CbEval { [Feature] public string[] Words { get; set; } } public class MyListener : VowpalWabbitBaseListener { private Test1 example; private Action action; public MyListener(Action action) { this.action = action; } public override void EnterExample(VowpalWabbitParser.ExampleContext context) { this.example = new Test1() { Features = new List>() }; } public override void ExitExample(VowpalWabbitParser.ExampleContext context) { this.example.Line = context.GetText(); this.action(this.example); } public override void ExitNumber(VowpalWabbitParser.NumberContext context) { context.value = float.Parse(context.GetText(), CultureInfo.InvariantCulture); } public override void ExitLabel_simple(VowpalWabbitParser.Label_simpleContext context) { this.example.Label = new SimpleLabel() { Label = context.value.value }; } public override void ExitFeatureSparse(VowpalWabbitParser.FeatureSparseContext context) { var index = context.index.Text; var x = context.x.value; if (index == "const") { this.example.Constant = x; } else { this.example.Features.Add(new KeyValuePair(index, x)); } } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/Test3.cs000066400000000000000000000043601323713302100211200ustar00rootroot00000000000000using System.IO; using System.Linq; using Microsoft.VisualStudio.TestTools.UnitTesting; using TrainSet0002Dat; using VW; namespace cs_unittest { [TestClass] public class Test3Class : TestBase { [TestMethod] [TestCategory("Vowpal Wabbit/Command line through marshalling")] public void Test3() { VWTestHelper.Learn( "-k train-sets/0002.dat -f models/0002.model --invariant", @"train-sets\0002.dat", @"train-sets\ref\0002.stderr"); } [TestMethod] [TestCategory("Vowpal Wabbit/Command line through marshalling")] public void Test4and6() { VWTestHelper.Learn( "-k -d train-sets/0002.dat -f models/0002.model --invariant", @"train-sets\0002.dat", @"train-sets\ref\0002.stderr"); VWTestHelper.Predict( "-k -t --invariant -i models/0002.model", @"train-sets\0002.dat", @"pred-sets\ref\0002b.predict"); } [TestMethod] [TestCategory("Vowpal Wabbit/Command line through marshalling")] public void Test5() { VWTestHelper.Learn( "-k --initial_t 1 --adaptive --invariant -q Tf -q ff -f models/0002a.model", @"train-sets\0002.dat", @"train-sets\ref\0002a.stderr"); VWTestHelper.Predict( "-k -t --invariant -i models/0002a.model", @"train-sets\0002.dat"); } [TestMethod] [TestCategory("Vowpal Wabbit/Command line through marshalling")] [Description("using normalized adaptive updates and a low --power_t")] public void Test7and8() { VWTestHelper.Learn( "-k --power_t 0.45 -f models/0002c.model", @"train-sets\0002.dat", @"train-sets\ref\0002c.stderr"); VWTestHelper.Predict( "-k -t -i models/0002c.model", @"train-sets\0002.dat", @"pred-sets\ref\0002c.predict"); } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestAllReduce.cs000066400000000000000000000151141323713302100226150ustar00rootroot00000000000000using cs_unittest.cbadf; using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Text; using System.Threading; using System.Threading.Tasks; using VW; using VW.Labels; using VW.Serializer; namespace cs_unittest { [TestClass] public class TestAllReduceClass : TestBase { private static void Ingest(VowpalWabbit vw, IEnumerable> blocks) { foreach (var block in blocks) { vw.Learn(block); } vw.EndOfPass(); } private static void Ingest(VowpalWabbitThreadedLearning vw, IEnumerable> blocks) { foreach (var block in blocks) { vw.Learn(block); } } private static void Ingest(VowpalWabbitAsync vw, IEnumerable, ContextualBanditLabel>> data) { foreach (var d in data) { vw.Learn(d.Item1, d.Item2, (int)d.Item3.Action, d.Item3); } } [TestMethod] [TestCategory("Vowpal Wabbit")] public async Task TestAllReduce() { var data = Enumerable.Range(1, 1000).Select(_ => Generator.GenerateShared(10)).ToList(); var stringSerializerCompiler = (VowpalWabbitSingleExampleSerializerCompiler) VowpalWabbitSerializerFactory.CreateSerializer(new VowpalWabbitSettings { EnableStringExampleGeneration = true }); var stringSerializerAdfCompiler = (VowpalWabbitSingleExampleSerializerCompiler) VowpalWabbitSerializerFactory.CreateSerializer(new VowpalWabbitSettings { EnableStringExampleGeneration = true }); var stringData = new List>(); VowpalWabbitPerformanceStatistics statsExpected; using (var spanningTree = new SpanningTreeClr()) { spanningTree.Start(); using (var vw1 = new VowpalWabbit(new VowpalWabbitSettings(@"--total 2 --node 1 --unique_id 0 --span_server localhost --cb_adf --rank_all --interact xy") { EnableStringExampleGeneration = true })) using (var vw2 = new VowpalWabbit(new VowpalWabbitSettings(@"--total 2 --node 0 --unique_id 0 --span_server localhost --cb_adf --rank_all --interact xy") { EnableStringExampleGeneration = true } )) { var stringSerializer = stringSerializerCompiler.Func(vw1); var stringSerializerAdf = stringSerializerAdfCompiler.Func(vw1); // serialize foreach (var d in data) { var block = new List(); using (var context = new VowpalWabbitMarshalContext(vw1)) { stringSerializer(context, d.Item1, SharedLabel.Instance); block.Add(context.ToString()); } block.AddRange(d.Item2.Select((a, i) => { using (var context = new VowpalWabbitMarshalContext(vw1)) { stringSerializerAdf(context, a, i == d.Item3.Action ? d.Item3 : null); return context.ToString(); } })); stringData.Add(block); } await Task.WhenAll( Task.Factory.StartNew(() => Ingest(vw1, stringData.Take(500))), Task.Factory.StartNew(() => Ingest(vw2, stringData.Skip(500)))); vw1.SaveModel("expected.1.model"); vw2.SaveModel("expected.2.model"); statsExpected = vw1.PerformanceStatistics; } } // skip header var expected1Model = File.ReadAllBytes("expected.1.model").Skip(0x15).ToList(); var expected2Model = File.ReadAllBytes("expected.2.model").Skip(0x15).ToList(); var settings = new VowpalWabbitSettings("--cb_adf --rank_all --interact xy") { ParallelOptions = new ParallelOptions { MaxDegreeOfParallelism = 2 }, ExampleCountPerRun = 2000, ExampleDistribution = VowpalWabbitExampleDistribution.RoundRobin }; using (var vw = new VowpalWabbitThreadedLearning(settings)) { await Task.WhenAll( Task.Factory.StartNew(() => Ingest(vw, stringData.Take(500))), Task.Factory.StartNew(() => Ingest(vw, stringData.Skip(500)))); // important to enqueue the request before Complete() is called var statsTask = vw.PerformanceStatistics; var modelSave = vw.SaveModel("actual.model"); await vw.Complete(); var statsActual = await statsTask; VWTestHelper.AssertEqual(statsExpected, statsActual); await modelSave; // skip header var actualModel = File.ReadAllBytes("actual.model").Skip(0x15).ToList(); CollectionAssert.AreEqual(expected1Model, actualModel); CollectionAssert.AreEqual(expected2Model, actualModel); } using (var vw = new VowpalWabbitThreadedLearning(settings)) { var vwManaged = vw.Create(); await Task.WhenAll( Task.Factory.StartNew(() => Ingest(vwManaged, data.Take(500))), Task.Factory.StartNew(() => Ingest(vwManaged, data.Skip(500)))); // important to enqueue the request before Complete() is called var statsTask = vw.PerformanceStatistics; var modelSave = vw.SaveModel("actual.managed.model"); await vw.Complete(); var statsActual = await statsTask; VWTestHelper.AssertEqual(statsExpected, statsActual); await modelSave; // skip header var actualModel = File.ReadAllBytes("actual.managed.model").Skip(0x15).ToList(); CollectionAssert.AreEqual(expected1Model, actualModel); CollectionAssert.AreEqual(expected2Model, actualModel); } } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestAntlr.cs000066400000000000000000000047341323713302100220430ustar00rootroot00000000000000using System.IO; using System.Text; using Microsoft.VisualStudio.TestTools.UnitTesting; using TrainSet0002Dat; using VW.Labels; namespace cs_unittest { [TestClass] public class TestAntlrClass { [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestAntlr() { var line1 = "0.521144 1 PFF/20091028|T PFF |f t1:-0.0236849 t5:-0.10215 r5:0.727735 t10:-0.0387662 r10:0.911208 t20:-0.00777943 r20:0.952668 t40:0.014542 r40:0.832479 t60:0.00395449 r60:0.724504 t90:0.0281418 r90:0.784653"; var listener = new DataListener(); listener.Created = (line, x, plabel) => { Assert.AreEqual(line1, line); Assert.AreEqual("PFF", x.T); var label = plabel as SimpleLabel; Assert.AreEqual(0.521144, label.Label, 1e-5); Assert.AreEqual(1, label.Weight); Assert.AreEqual(13, x.F.Count); Assert.AreEqual("t1", x.F[0].Key); Assert.AreEqual(-0.0236849, x.F[0].Value, 1e-5); }; VWTestHelper.ParseInput(new MemoryStream(Encoding.UTF8.GetBytes(line1)), listener); var line2 = "1 |f 13:3.9656971e-02 24:3.4781646e-02 69:4.6296168e-02 85:6.1853945e-02 140:3.2349996e-02 156:1.0290844e-01 175:6.8493910e-02 188:2.8366476e-02 229:7.4871540e-02 230:9.1505975e-02 234:5.4200061e-02 236:4.4855952e-02 238:5.3422898e-02 387:1.4059304e-01 394:7.5131744e-02 433:1.1118756e-01 434:1.2540409e-01 438:6.5452829e-02 465:2.2644201e-01 468:8.5926279e-02 518:1.0214076e-01 534:9.4191484e-02 613:7.0990764e-02 646:8.7701865e-02 660:7.2289191e-02 709:9.0660661e-02 752:1.0580081e-01 757:6.7965068e-02 812:2.2685185e-01 932:6.8250686e-02 1028:4.8203137e-02 1122:1.2381379e-01 1160:1.3038123e-01 1189:7.1542501e-02 1530:9.2655659e-02 1664:6.5160148e-02 1865:8.5823394e-02 2524:1.6407280e-01 2525:1.1528353e-01 2526:9.7131468e-02 2536:5.7415009e-01 2543:1.4978983e-01 2848:1.0446861e-01 3370:9.2423186e-02 3960:1.5554591e-01 7052:1.2632671e-01 16893:1.9762035e-01 24036:3.2674628e-01 24303:2.2660980e-010"; listener.Created = (line, x, plabel) => { Assert.AreEqual(line2, line); var label = plabel as SimpleLabel; Assert.AreEqual(1, label.Label, 1e-5); Assert.AreEqual(49, x.F.Count); }; VWTestHelper.ParseInput(new MemoryStream(Encoding.UTF8.GetBytes(line2)), listener); } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestArguments.cs000066400000000000000000000125771323713302100227340ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Text; using System.Threading.Tasks; using VW; namespace cs_unittest { [TestClass] public class TestArgumentsClass { [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestArguments() { using (var vw = new VowpalWabbit(new VowpalWabbitSettings("--cb_explore_adf --epsilon 0.3 --interact ud") { Verbose = true })) { // --cb_explore_adf --epsilon 0.3 --interact ud --cb_adf--csoaa_ldf multiline --csoaa_rank Console.WriteLine(vw.Arguments.CommandLine); Assert.IsTrue(vw.Arguments.CommandLine.Contains("--cb_explore_adf")); Assert.IsTrue(vw.Arguments.CommandLine.Contains("--epsilon 0.3")); Assert.IsTrue(vw.Arguments.CommandLine.Contains("--interact ud")); Assert.IsTrue(vw.Arguments.CommandLine.Contains("--csoaa_ldf multiline")); Assert.IsTrue(vw.Arguments.CommandLine.Contains("--csoaa_rank")); vw.SaveModel("args.model"); } using (var vw = new VowpalWabbit(new VowpalWabbitSettings { ModelStream = File.Open("args.model", FileMode.Open) })) { Console.WriteLine(vw.Arguments.CommandLine); // --no_stdin--bit_precision 18--cb_explore_adf--epsilon 0.300000--cb_adf--cb_type ips --csoaa_ldf multiline--csoaa_rank--interact ud Assert.IsTrue(vw.Arguments.CommandLine.Contains("--no_stdin")); Assert.IsTrue(vw.Arguments.CommandLine.Contains("--bit_precision 18")); Assert.IsTrue(vw.Arguments.CommandLine.Contains("--cb_explore_adf")); Assert.IsTrue(vw.Arguments.CommandLine.Contains("--epsilon 0.3")); Assert.IsTrue(vw.Arguments.CommandLine.Contains("--interact ud")); Assert.IsTrue(vw.Arguments.CommandLine.Contains("--csoaa_ldf multiline")); Assert.IsTrue(vw.Arguments.CommandLine.Contains("--csoaa_rank")); Assert.IsTrue(vw.Arguments.CommandLine.Contains("--cb_type ips")); Assert.IsTrue(vw.Arguments.CommandLine.Contains("--csoaa_ldf multiline")); Assert.IsTrue(vw.Arguments.CommandLine.Contains("--interact ud")); Assert.IsTrue(vw.Arguments.CommandLine.Contains("--csoaa_rank")); } } [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestQuietAndTestArguments() { using (var vw = new VowpalWabbit("--quiet -t")) { vw.SaveModel("args.model"); } using (var vw = new VowpalWabbitModel(new VowpalWabbitSettings { ModelStream = File.Open("args.model", FileMode.Open) })) { Assert.IsFalse(vw.Arguments.CommandLine.Contains("--quiet")); Assert.IsTrue(vw.Arguments.CommandLine.Contains("-t")); using (var vwSub = new VowpalWabbit(new VowpalWabbitSettings { Model = vw })) { Assert.IsTrue(vwSub.Arguments.CommandLine.Contains("--quiet")); Assert.IsTrue(vwSub.Arguments.CommandLine.Contains("-t")); } } using (var vw = new VowpalWabbit("")) { vw.SaveModel("args.model"); } using (var vw = new VowpalWabbitModel(new VowpalWabbitSettings { ModelStream = File.Open("args.model", FileMode.Open) })) { Assert.IsFalse(vw.Arguments.CommandLine.Contains("--quiet")); Assert.IsTrue(vw.Arguments.CommandLine.Contains("-t")); using (var vwSub = new VowpalWabbit(new VowpalWabbitSettings { Model = vw })) { Assert.IsTrue(vwSub.Arguments.CommandLine.Contains("--quiet")); Assert.IsTrue(vwSub.Arguments.CommandLine.Contains("-t")); } } } [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestArgumentDeDup() { using (var vw = new VowpalWabbit("-l 0.3 -l 0.3 --learning_rate 0.3 -f model1 --save_resume -q ab")) { Assert.AreEqual(0.3f, vw.Native.Arguments.LearningRate); } try { using (var vw = new VowpalWabbit("-l 0.3 -l 0.3 --learning_rate 0.1 -f model1 --save_resume -q ab")) { Assert.AreEqual(0.3f, vw.Native.Arguments.LearningRate); } Assert.Fail("Disagreering arguments not detected"); } catch (VowpalWabbitException) { } using (var vw = new VowpalWabbit("-i model1 --save_resume")) { Assert.AreEqual(0.5f, vw.Native.Arguments.LearningRate); } using (var vw = new VowpalWabbit("-i model1 --save_resume -q ab -l 0.4")) { Assert.AreEqual(0.4f, vw.Native.Arguments.LearningRate); } // make sure different representations of arguments are matched using (var vw = new VowpalWabbit("--cb_explore_adf --epsilon 0.1 -f model2")) { } using (var vw = new VowpalWabbit("--cb_explore_adf --epsilon 0.1000 -i model2")) { } } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestAzure.cs000066400000000000000000000610371323713302100220500ustar00rootroot00000000000000using Microsoft.ApplicationInsights.DataContracts; using Microsoft.ApplicationInsights.Extensibility; using Microsoft.ServiceBus.Messaging; using Microsoft.VisualStudio.TestTools.UnitTesting; using Microsoft.WindowsAzure.Storage; using Microsoft.WindowsAzure.Storage.Blob; using MoreLinq; using Newtonsoft.Json; using System; using System.Collections.Generic; using System.Diagnostics; using System.Globalization; using System.IO; using System.Linq; using System.Text; using System.Text.RegularExpressions; using System.Threading.Tasks; using VW; using VW.Azure.Trainer; using VW.Azure.Trainer.Checkpoint; using VW.Serializer; namespace cs_unittest { [TestClass] public class TestAzure { public class SharedFeatures { public string Location { get; set; } } public class ActionNamespace { public float Category { get; set; } } public class ActionFeatures { [JsonProperty("b")] public ActionNamespace Namespace { get; set; } // TODO: _tag } public class Context { // TODO: _ProbabilityOfDrop [JsonIgnore] public int Index { get; set; } [JsonIgnore] public string JSON { get { return JsonConvert.SerializeObject(this); } } [JsonIgnore] public byte[] JSONAsBytes { get { return Encoding.UTF8.GetBytes(this.JSON); } } [JsonProperty("_eventid")] public string EventId { get; set; } [JsonProperty("_timestamp")] public DateTime Timestamp { get; set; } [JsonProperty("_a")] public int[] ActionIndicies { get; set; } [JsonProperty("_p")] public float[] Probabilities { get; set; } [JsonProperty("_label_action")] public int LabelAction { get; set; } [JsonProperty("_label_cost")] public float LabelCost { get; set; } [JsonProperty("_label_probability")] public float LabelProbability { get; set; } [JsonProperty("_labelindex")] public int LabelIndex { get; set; } [JsonProperty("a")] public SharedFeatures Shared { get; set; } [JsonProperty("_multi")] public ActionFeatures[] Actions { get; set; } } private static string GetConfiguration(string name) { var value = Environment.GetEnvironmentVariable(name); if (!string.IsNullOrEmpty(value)) return value.Trim(); var path = Directory.GetCurrentDirectory(); do { var filename = Path.Combine(path, "vw_azure.config"); if (File.Exists(filename)) { var q = from line in File.ReadAllLines(filename) let m = Regex.Match(line, @"^(\S+)\s*=(.*)$") where m.Success where m.Groups[1].Value == name select m.Groups[2].Value; value = q.FirstOrDefault(); if (!string.IsNullOrEmpty(value)) return value.Trim(); } var di = Directory.GetParent(path); if (di == null) Assert.Fail($"Configuration variable '{name}' not found. Search for environment variable or vw_azure.config"); path = di.FullName; } while (true); } private class OnlineTrainerWrapper : IDisposable { internal string trainArguments; internal OnlineTrainerBlobs Blobs; private string storageConnectionString = GetConfiguration("storageConnectionString"); private string inputEventHubConnectionString = GetConfiguration("inputEventHubConnectionString"); private string evalEventHubConnectionString = GetConfiguration("evalEventHubConnectionString"); private SynchronizedCollection exceptions; private LearnEventProcessorHost trainProcesserHost; internal OnlineTrainerWrapper(string trainArguments) { this.trainArguments = trainArguments; Blobs = new OnlineTrainerBlobs(storageConnectionString); // register with AppInsights to collect exceptions // need to set the instrumentation key, otherwise the processor is ignored. TelemetryConfiguration.Active.InstrumentationKey = "00000000-0000-0000-0000-000000000000"; exceptions = new SynchronizedCollection(); var builder = TelemetryConfiguration.Active.TelemetryProcessorChainBuilder; builder.Use((next) => new TestTelemetryProcessor(next, exceptions)); builder.Build(); } void AssertNoExceptionsThroughAppInsights() { Assert.AreEqual(0, exceptions.Count, string.Join("\n", exceptions.Select(e => e.Exception.Message + " " + e.Message))); } internal async Task StartAsync(ICheckpointPolicy checkpointPolicy) { trainProcesserHost = new LearnEventProcessorHost(); await trainProcesserHost.StartAsync(new OnlineTrainerSettingsInternal { CheckpointPolicy = checkpointPolicy, JoinedEventHubConnectionString = inputEventHubConnectionString, EvalEventHubConnectionString = evalEventHubConnectionString, StorageConnectionString = storageConnectionString, Metadata = new OnlineTrainerSettings { ApplicationID = "vwunittest", TrainArguments = trainArguments }, EnableExampleTracing = false, EventHubStartDateTimeUtc = DateTime.UtcNow // ignore any events that arrived before this time }); AssertNoExceptionsThroughAppInsights(); } internal async Task PollTrainerCheckpoint(Predicate predicate) { // wait for trainer to checkpoint await Blobs.PollTrainerCheckpoint(exceptions, predicate); } internal void SendData(IEnumerable data) { // send events to event hub var eventHubInputClient = EventHubClient.CreateFromConnectionString(inputEventHubConnectionString); data.ForEach(c => eventHubInputClient.Send(new EventData(c.JSONAsBytes) { PartitionKey = c.Index.ToString() })); } public void Dispose() { if (trainProcesserHost != null) { trainProcesserHost.Dispose(); trainProcesserHost = null; } } internal void TrainOffline(string message, string modelId, Dictionary data, IEnumerable eventOrder, Uri onlineModelUri, string trainArguments = null) { // allow override if (trainArguments == null) trainArguments = this.trainArguments; // train model offline using trackback var settings = new VowpalWabbitSettings(trainArguments + $" --id {modelId} --save_resume --preserve_performance_counters -f offline.model"); using (var vw = new VowpalWabbitJson(settings)) { foreach (var id in eventOrder) { var json = data[id].JSON; var progressivePrediction = vw.Learn(json, VowpalWabbitPredictionType.ActionProbabilities); // TODO: validate eval output } } using (var vw = new VowpalWabbit("-i offline.model --save_resume --readable_model offline.model.txt -f offline.reset_perf_counters.model")) { } Blobs.DownloadFile(onlineModelUri, "online.model"); using (var vw = new VowpalWabbit("-i online.model --save_resume --readable_model online.model.txt -f online.reset_perf_counters.model")) { } // validate that the model is the same CollectionAssert.AreEqual( File.ReadAllBytes("offline.reset_perf_counters.model"), File.ReadAllBytes("online.reset_perf_counters.model"), $"{message}. Offline and online model differs. Compare online.model.txt with offline.model.txt to compare"); } } private async Task RunTrainer(string args, IEnumerable data, Dictionary dataMap, int expectedNumStates, bool cleanBlobs) { var trainer = new OnlineTrainerWrapper("--cb_explore_adf --epsilon 0.1 -q ab -l 0.1"); if (cleanBlobs) trainer.Blobs.Cleanup().Wait(); // start listening for event hub await trainer.StartAsync(new CountingCheckpointPolicy(100)); // send data to event hub trainer.SendData(data); await trainer.PollTrainerCheckpoint(blobs => blobs.ModelBlobs.Count == expectedNumStates && blobs.ModelTrackbackBlobs.Count == expectedNumStates && blobs.StateJsonBlobs.Count == expectedNumStates); // download & parse trackback file trainer.Blobs.DownloadTrackbacksOrderedByTime(); foreach (var trackback in trainer.Blobs.Trackbacks) // due to checkpoint policy = 100 Assert.AreEqual(100, trackback.EventIds.Count, $"{trackback.Blob.Uri} does not contain the expected 100 events. Actual: {trackback.EventIds.Count}"); return trainer; } [TestMethod] [TestCategory("NotOnVSO")] [TestCategory("Vowpal Wabbit")] [Ignore] public async Task TestAzureTrainerRestart() { // generate data var data = GenerateData(600).ToList(); var dataMap = data.ToDictionary(d => d.EventId, d => d); var args = "--cb_explore_adf --epsilon 0.1 -q ab -l 0.1"; using (var trainer = await RunTrainer(args, data.Take(220), dataMap, expectedNumStates: 2, cleanBlobs: true)) { trainer.TrainOffline("produce the 1st model", trainer.Blobs.Trackbacks[0].ModelId, dataMap, trainer.Blobs.Trackbacks[0].EventIds, trainer.Blobs.ModelBlobs[0].Uri); // keep model for subsequent training File.Copy("offline.model", "split1.model", overwrite: true); trainer.TrainOffline("produce the 2nd model by training through all events", trainer.Blobs.Trackbacks[1].ModelId, dataMap, trainer.Blobs.Trackbacks.SelectMany(t => t.EventIds), trainer.Blobs.ModelBlobs[1].Uri); File.Copy("offline.model", "split2.model", overwrite: true); trainer.TrainOffline("produce the 2nd model by starting from the 1st and then continuing", trainer.Blobs.Trackbacks[1].ModelId, dataMap, trainer.Blobs.Trackbacks[1].EventIds, trainer.Blobs.ModelBlobs[1].Uri, "-i split1.model -l 0.1"); } // restart trainer and resume from split2.model, covering "fresh -> load" transition using (var trainer = await RunTrainer(args, data.Skip(220).Take(120), dataMap, expectedNumStates: 3, cleanBlobs: false)) { var lastTrackback = trainer.Blobs.Trackbacks.Last(); var lastBlob = trainer.Blobs.ModelBlobs.Last(); trainer.TrainOffline("produce the 3rd model by training through all events", lastTrackback.ModelId, dataMap, trainer.Blobs.Trackbacks.SelectMany(t => t.EventIds), lastBlob.Uri); trainer.TrainOffline("produce the 3rd model by starting from the 2nd and then continuing", lastTrackback.ModelId, dataMap, lastTrackback.EventIds, lastBlob.Uri, "-i split2.model -l 0.1"); File.Copy("offline.model", "split3.model", overwrite: true); } // restart ones more to cover "load -> save" using (var trainer = await RunTrainer(args, data.Skip(340).Take(120), dataMap, expectedNumStates: 4, cleanBlobs: false)) { var lastTrackback = trainer.Blobs.Trackbacks.Last(); var lastBlob = trainer.Blobs.ModelBlobs.Last(); trainer.TrainOffline("produce the 4th model by training through all events", lastTrackback.ModelId, dataMap, trainer.Blobs.Trackbacks.SelectMany(t => t.EventIds), lastBlob.Uri); trainer.TrainOffline("produce the 4th model by starting from the 3rd and then continuing", lastTrackback.ModelId, dataMap, lastTrackback.EventIds, lastBlob.Uri, "-i split3.model -l 0.1"); } } [TestMethod] [TestCategory("NotOnVSO")] [TestCategory("Vowpal Wabbit")] [Ignore] public async Task TestAzureTrainer() { using (var trainer = new OnlineTrainerWrapper("--cb_explore_adf --epsilon 0.2 -q ab")) { trainer.Blobs.Cleanup().Wait(); // generate data var data = GenerateData(100).ToList(); var dataMap = data.ToDictionary(d => d.EventId, d => d); // start listening for event hub await trainer.StartAsync(new CountingCheckpointPolicy(data.Count)); // send data to event hub trainer.SendData(data); // wait for trainer to checkpoint await trainer.PollTrainerCheckpoint(blobs => blobs.ModelBlobs.Count > 0 && blobs.ModelTrackbackBlobs.Count > 0 && blobs.StateJsonBlobs.Count > 0 ); // download & parse trackback file trainer.Blobs.DownloadTrackbacksOrderedByTime(); Assert.AreEqual(1, trainer.Blobs.Trackbacks.Count); var trackback = trainer.Blobs.Trackbacks[0]; Assert.AreEqual(data.Count, trackback.EventIds.Count); Assert.AreEqual(1, trainer.Blobs.ModelBlobs.Count); trainer.TrainOffline("train a model for this set of events", trackback.ModelId, dataMap, trackback.EventIds, trainer.Blobs.ModelBlobs[0].Uri); } } [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestCbProgressiveValidation() { int numExamples = 1024; foreach (var cbType in new[] { "ips", "dr", "mtr" }) { var trainArguments = $"--cb_explore_adf --epsilon 0.1 --bag 3 -q ab --power_t 0 -l 0.1 --cb_type {cbType} --random_seed 50"; int[] topActionCounts = new int[3]; using (var vw1 = new VowpalWabbitJson(trainArguments)) using (var vw2 = new VowpalWabbitJson(trainArguments)) { foreach (var ex in GenerateData(numExamples)) { var json = ex.JSON; var pred1_a = vw1.Predict(json, VowpalWabbitPredictionType.ActionProbabilities); var pred1_b = vw1.Learn(json, VowpalWabbitPredictionType.ActionProbabilities); var pred2 = vw2.Learn(json, VowpalWabbitPredictionType.ActionProbabilities); AreEqual(pred1_a, pred2, cbType); AreEqual(pred1_b, pred2, cbType); topActionCounts[pred2[0].Action]++; //Debug.WriteLine(json); //Debug.WriteLine("Prob1.pred: " + string.Join(",", pred1_a.Select(a=>$"{a.Action}:{a.Score}"))); //Debug.WriteLine("Prob1.learn: " + string.Join(",", pred1_b.Select(a=>$"{a.Action}:{a.Score}"))); //Debug.WriteLine("Prob2.learn: " + string.Join(",", pred2.Select(a=>$"{a.Action}:{a.Score}"))); //Debug.WriteLine(""); } } foreach (var count in topActionCounts) Assert.IsTrue(count < numExamples * 0.8, $"Unexpected action distribution: {count}"); Debug.WriteLine($"cb_types: {cbType} " + string.Join(",", topActionCounts.Select((count, i) => $"{i}:{count}"))); } } private static void AreEqual(ActionScore[] expected, ActionScore[] actual, string cbType) { Assert.AreEqual(expected.Length, actual.Length); for (int i = 0; i < expected.Length; i++) { Assert.AreEqual(expected[i].Action, actual[i].Action, $"cb_type: {cbType} Action mismatch at index {i}. Expected: {expected[i].Action} Actual: {actual[i].Action}"); Assert.AreEqual(expected[i].Score, actual[i].Score, $"cb_type: {cbType} Score mismatch at index {i}. Expected: {expected[i].Score} Actual: {actual[i].Score}"); } } internal class Trackback { public IListBlobItem Blob; public string ModelId; public List EventIds; public DateTime Timestamp; } private static IEnumerable GenerateData(int n) { var random = new Random(42); var locations = new[] { "east", "west" }; for (int i = 0; i < n; i++) { var action = random.Next(2); // (i % 2); var prob = (float)random.NextDouble(); var probs = action == 0 ? new[] { prob, 1 - prob } : new[] { 1 - prob, prob }; yield return new Context { Index = i, ActionIndicies = new[] { 1, 2 }, Probabilities = new[] { 0.5f, 0.5f },// probs, Timestamp = DateTime.UtcNow, EventId = Guid.NewGuid().ToString("n"), LabelAction = action + 1, // random.Next(1, 3), //action + 1, LabelCost = random.Next(4) - 2, LabelIndex = action, // action, LabelProbability = prob, Shared = new SharedFeatures { Location = locations[random.Next(2)] }, Actions = new[] { new ActionFeatures { Namespace = new ActionNamespace { Category = (float)random.NextDouble() } }, new ActionFeatures { Namespace = new ActionNamespace { Category = 1.5f + (float)random.NextDouble() } }, new ActionFeatures { Namespace = new ActionNamespace { Category = 1.5f + (float)random.NextDouble() } } //new ActionFeatures { Namespace = new ActionNamespace { Category = "games" } }, //new ActionFeatures { Namespace = new ActionNamespace { Category = "news" } } } }; } } internal class OnlineTrainerBlobs { internal CloudBlobClient BlobClient; internal CloudBlobContainer ModelContainer; internal CloudBlockBlob CurrentModel; internal CloudBlobContainer TrainerContainer; internal List ModelBlobs; internal List ModelTrackbackBlobs; internal List StateJsonBlobs; internal List Trackbacks; internal OnlineTrainerBlobs(string storageConnectionString) { this.BlobClient = CloudStorageAccount.Parse(storageConnectionString).CreateCloudBlobClient(); this.ModelContainer = this.BlobClient.GetContainerReference("mwt-models"); this.CurrentModel = this.ModelContainer.GetBlockBlobReference("current"); this.TrainerContainer = this.BlobClient.GetContainerReference("onlinetrainer"); } internal void DownloadFile(Uri uri, string filename) { new CloudBlob(uri, BlobClient.Credentials).DownloadToFile(filename, FileMode.Create); } internal async Task Cleanup() { // don't delete the container as this will trigger a conflict unless we wait... if (this.ModelContainer.Exists()) this.CurrentModel.DeleteIfExists(); if (this.TrainerContainer.Exists()) { // don't delete the container as this will trigger a conflict unless we wait... await Task.WhenAll( this.TrainerContainer.ListBlobs(useFlatBlobListing: true) .Select(blob => new CloudBlob(blob.Uri, this.BlobClient.Credentials).DeleteIfExistsAsync())); } } internal async Task PollTrainerCheckpoint(SynchronizedCollection exceptions, Predicate predicate) { // wait for files to show up for (int i = 0; i < 30; i++) { await Task.Delay(TimeSpan.FromSeconds(1)); if (exceptions.Count > 0) Assert.Fail(string.Join(";", exceptions.Select(e => e.Message))); // mwt-models if (!this.ModelContainer.Exists()) continue; // mwt-models/current if (!this.CurrentModel.Exists()) continue; // onlinetrainer if (!this.TrainerContainer.Exists()) continue; // onlinetrainer/20161128/002828 var blobs = this.TrainerContainer.ListBlobs(useFlatBlobListing: true); this.ModelBlobs = blobs.Where(b => b.Uri.ToString().EndsWith("model")) .OrderBy(uri => DateTime.ParseExact(uri.Parent.Prefix, "yyyyMMdd/HHmmss/", CultureInfo.InvariantCulture)) .ToList(); this.ModelTrackbackBlobs = blobs.Where(b => b.Uri.ToString().EndsWith("model.trackback")).ToList(); this.StateJsonBlobs = blobs.Where(b => !string.IsNullOrEmpty(b.Parent.Prefix) && b.Uri.ToString().EndsWith("state.json")).ToList(); if (predicate(this)) return; } Assert.Fail("Trainer didn't produce checkpoints"); } internal void DownloadTrackbacksOrderedByTime() { this.Trackbacks = this.ModelTrackbackBlobs.Select(b => { var trackbackStr = this.DownloadNonEmptyBlob(b); var trackback = trackbackStr.Split('\n'); // modelid: faf5e313-46bb-4852-af05-576c3a1c2c67 var m = Regex.Match(trackback[0], "^modelid: (.+)$"); Assert.IsTrue(m.Success, $"Unable to extract model id from trackback file. Line '{trackback[0]}'"); return new Trackback { Blob = b, Timestamp = DateTime.ParseExact(b.Parent.Prefix, "yyyyMMdd/HHmmss/", CultureInfo.InvariantCulture), ModelId = m.Groups[1].Value, EventIds = trackback.Skip(1).ToList() }; }) .OrderBy(x => x.Timestamp).ToList(); } internal string DownloadNonEmptyBlob(IListBlobItem blob) { using (var memStream = new MemoryStream()) { new CloudBlob(blob.Uri, this.BlobClient.Credentials).DownloadToStream(memStream); var content = Encoding.UTF8.GetString(memStream.ToArray()); Assert.IsTrue(!string.IsNullOrWhiteSpace(content), $"File is empty: '{blob.Uri}'"); return content; } } } //[TestMethod] //public void TestQueueDictionary() //{ // var qd = new QueueDictionary(); // Assert.IsNull(qd.Remove(1)); // qd.Enqueue(1, "foo"); // Assert.AreEqual("foo", qd.Remove(1)); // Assert.AreEqual(0, qd.DequeueIf(_ => true).Count()); // qd.Enqueue(1, "foo"); // qd.Enqueue(2, "bar"); // Assert.AreEqual(1, qd.DequeueIf(key => key == "foo").Count()); // Assert.AreEqual("foo", qd.Remove(1)); //} } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestBase.cs000066400000000000000000000021511323713302100216240ustar00rootroot00000000000000using System.IO; using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Linq; using VW; using System.Text.RegularExpressions; using System.Runtime.InteropServices; using System.Diagnostics; namespace cs_unittest { [TestClass] public abstract class TestBase : IDisposable { public TestBase() { this.Init(); } [TestInitialize] public void Init() { var basePath = Path.GetDirectoryName(typeof(TestBase).Assembly.Location); Environment.CurrentDirectory = basePath + @"\..\..\..\test"; if (!Directory.Exists("models")) { Directory.CreateDirectory("models"); } } public void Dispose() { try { if (Directory.Exists("models")) { Directory.Delete("models", true); } } catch (Exception ex) { Console.WriteLine("Failed to delete model directory: "+ex.Message); } } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestCbAdf.cs000066400000000000000000000474311323713302100217230ustar00rootroot00000000000000using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Threading.Tasks; using Microsoft.VisualStudio.TestTools.UnitTesting; using VW; using VW.Labels; using VW.Serializer.Attributes; using Newtonsoft.Json; namespace cs_unittest { [TestClass] public class TestCbAdfClass : TestBase { public void ProfilePerformanceWithStringData() { string outModelFile = "profile_cb_adf.model"; using (var vw = new VowpalWabbit("--cb_adf --rank_all")) { DataString[] sampleData = CreateStringCbAdfData(1000 * 1000); foreach (DataString example in sampleData) { vw.Learn(example, example.ActionDependentFeatures, example.SelectedActionIndex, example.Label); } vw.Native.SaveModel(outModelFile); } File.Delete(outModelFile); } public void ProfilePerformanceWithFloatData() { string outModelFile = "profile_cb_adf.model"; using (var vw = new VowpalWabbit("--cb_adf --rank_all")) { DataFloat[] sampleData = CreateFloatCbAdfData(1000 * 1000); foreach (DataFloat example in sampleData) { vw.Learn(example, example.ActionDependentFeatures, example.SelectedActionIndex, example.Label); } vw.Native.SaveModel(outModelFile); } File.Delete(outModelFile); } private void Validate(VowpalWabbitExampleValidator vwSharedValidation, VowpalWabbitExampleValidator vwADFValidation, DataString example) { vwSharedValidation.Validate(example.Line, example, SharedLabel.Instance); for (int i = 0; i < example.ActionDependentFeatures.Count; i++) { var adf = example.ActionDependentFeatures[i]; vwADFValidation.Validate(adf.Line, adf, i == example.SelectedActionIndex ? example.Label : null); } } public static void TestMemoryLeak() { string outModelFile = "cb_adf_mem_leak.model"; using (var vw = new VowpalWabbit("--cb_adf --rank_all")) { DataString[] sampleData = CreateStringCbAdfData(1000); foreach (DataString example in sampleData) { vw.Learn(example, example.ActionDependentFeatures, example.SelectedActionIndex, example.Label); } vw.Native.SaveModel(outModelFile); } var vwModel = new VowpalWabbitModel(new VowpalWabbitSettings(string.Format("--quiet -t -i {0}", outModelFile)) { MaxExampleCacheSize = 1024 }); var pool = new VowpalWabbitThreadedPrediction(vwModel); while (true) { vwModel = new VowpalWabbitModel(new VowpalWabbitSettings(string.Format("--quiet -t -i {0}", outModelFile)) { MaxExampleCacheSize = 1024 }); pool.UpdateModel(vwModel); } } [TestMethod] [TestCategory("Vowpal Wabbit/Command line through marshalling")] public void Test87() { using (var vw = new VowpalWabbit("--cb_adf --rank_all")) using (var vwSharedValidation = new VowpalWabbitExampleValidator("--cb_adf --rank_all")) using (var vwADFValidation = new VowpalWabbitExampleValidator("--cb_adf --rank_all")) { var sampleData = CreateSampleCbAdfData(); var example = sampleData[0]; Validate(vwSharedValidation, vwADFValidation, example); var result = vw.LearnAndPredict(example, example.ActionDependentFeatures, example.SelectedActionIndex, example.Label); ReferenceEquals(example.ActionDependentFeatures[0], result[0]); ReferenceEquals(example.ActionDependentFeatures[1], result[1]); ReferenceEquals(example.ActionDependentFeatures[2], result[2]); example = sampleData[1]; Validate(vwSharedValidation, vwADFValidation, example); result = vw.LearnAndPredict(example, example.ActionDependentFeatures, example.SelectedActionIndex, example.Label); ReferenceEquals(example.ActionDependentFeatures[0], result[1]); ReferenceEquals(example.ActionDependentFeatures[1], result[0]); example = sampleData[2]; Validate(vwSharedValidation, vwADFValidation, example); result = vw.Predict(example, example.ActionDependentFeatures); ReferenceEquals(example.ActionDependentFeatures[0], result[1]); ReferenceEquals(example.ActionDependentFeatures[1], result[0]); } } [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestSharedModel() { string cbadfModelFile = "models/cb_adf.model"; var sampleData = CreateSampleCbAdfData(); using (var vw = new VowpalWabbit("--cb_adf --rank_all")) using (var vwSharedValidation = new VowpalWabbitExampleValidator("--cb_adf --rank_all")) using (var vwADFValidation = new VowpalWabbitExampleValidator("--cb_adf --rank_all")) { foreach (DataString example in sampleData) { Validate(vwSharedValidation, vwADFValidation, example); vw.Learn(example, example.ActionDependentFeatures, example.SelectedActionIndex, example.Label); } vw.Native.SaveModel(cbadfModelFile); } // Get ground truth predictions var expectedPredictions = new List(); using (var vw = new VowpalWabbit(string.Format("-t -i {0}", cbadfModelFile))) { foreach (DataString example in sampleData) { var pred = vw.Predict(example, example.ActionDependentFeatures); if (pred == null) expectedPredictions.Add(null); else { expectedPredictions.Add(pred.Select(p => p.Feature).ToArray()); } } } // Test synchronous VW instances using shared model using (var vwModel = new VowpalWabbitModel(new VowpalWabbitSettings("-t") { ModelStream = File.OpenRead(cbadfModelFile) })) using (var vwShared1 = new VowpalWabbit(new VowpalWabbitSettings{ Model = vwModel })) using (var vwShared2 = new VowpalWabbit(new VowpalWabbitSettings{ Model = vwModel })) { for (int i = 0; i < sampleData.Length; i++) { var actualPrediction = vwShared1.Predict(sampleData[i], sampleData[i].ActionDependentFeatures); if (actualPrediction == null) ReferenceEquals(expectedPredictions[i], actualPrediction); else ReferenceEquals(expectedPredictions[i], actualPrediction.Select(p => p.Feature).ToArray()); } } // Test concurrent VW instances using shared model and model pool using (var vwModel = new VowpalWabbitModel(new VowpalWabbitSettings("-t") { ModelStream = File.OpenRead(cbadfModelFile) })) using (var vwPool = new VowpalWabbitThreadedPrediction(vwModel)) { Parallel.For ( fromInclusive: 0, toExclusive: 20, parallelOptions: new ParallelOptions { MaxDegreeOfParallelism = Environment.ProcessorCount * 2 }, body: i => { using (var vwObject = vwPool.GetOrCreate()) { var actualPredictions = new List(); foreach (DataString example in sampleData) { actualPredictions.Add(vwObject.Value.Predict(example, example.ActionDependentFeatures).Select(p => p.Feature).ToArray()); } Assert.AreEqual(expectedPredictions.Count, actualPredictions.Count); for (int j = 0; j < expectedPredictions.Count; j++) { ReferenceEquals(expectedPredictions[j], actualPredictions[j]); } } } ); } } [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestCbAdfExplore() { var json = JsonConvert.SerializeObject(new { U = new { age = "18" }, _multi = new[] { new { G = new { _text = "this rocks" }, K = new { constant = 1, doc = "1" } }, new { G = new { _text = "something NYC" }, K = new { constant = 1, doc = "2" } }, }, _label_Action = 2, _label_Probability = 0.1, _label_Cost = -1, _labelIndex = 1 }); using (var vw = new VowpalWabbitJson("--cb_explore_adf --bag 4 --epsilon 0.0001 --cb_type mtr --marginal K -q UG -b 24 --power_t 0 --l1 1e-9 -l 4e-3")) { for (int i = 0; i < 50; i++) { var pred = vw.Learn(json, VowpalWabbitPredictionType.ActionProbabilities); Assert.AreEqual(2, pred.Length); if (i > 40) { Assert.AreEqual(1, (int)pred[0].Action); Assert.IsTrue(pred[0].Score > .9); Assert.AreEqual(0, (int)pred[1].Action); Assert.IsTrue(pred[1].Score < .1); } } vw.Native.SaveModel("cbadfexplore.model"); } using (var vw = new VowpalWabbitJson(new VowpalWabbitSettings { Arguments = "-t", ModelStream = File.Open("cbadfexplore.model", FileMode.Open) })) { var predObj = vw.Predict(json, VowpalWabbitPredictionType.Dynamic); Assert.IsInstanceOfType(predObj, typeof(ActionScore[])); var pred = (ActionScore[])predObj; Assert.AreEqual(1, (int)pred[0].Action); Assert.IsTrue(pred[0].Score > .9); Assert.AreEqual(0, (int)pred[1].Action); Assert.IsTrue(pred[1].Score < .1); } using (var vwModel = new VowpalWabbitModel(new VowpalWabbitSettings { ModelStream = File.Open("cbadfexplore.model", FileMode.Open) })) using (var vwSeeded = new VowpalWabbitJson(new VowpalWabbitSettings { Model = vwModel })) { var pred = vwSeeded.Predict(json, VowpalWabbitPredictionType.ActionProbabilities); Assert.AreEqual(1, (int)pred[0].Action); Assert.IsTrue(pred[0].Score > .9); Assert.AreEqual(0, (int)pred[1].Action); Assert.IsTrue(pred[1].Score < .1); } using (var vwModel = new VowpalWabbitModel(new VowpalWabbitSettings { ModelStream = File.Open("cbadfexplore.model", FileMode.Open) })) { using (var vwPool = new VowpalWabbitJsonThreadedPrediction(vwModel)) using (var vw = vwPool.GetOrCreate()) { var predObj = vw.Value.Predict(json, VowpalWabbitPredictionType.Dynamic); Assert.IsInstanceOfType(predObj, typeof(ActionScore[])); var pred = (ActionScore[])predObj; Assert.AreEqual(1, (int)pred[0].Action); Assert.IsTrue(pred[0].Score > .9); Assert.AreEqual(0, (int)pred[1].Action); Assert.IsTrue(pred[1].Score < .1); } } } private DataString[] CreateSampleCbAdfData() { var sampleData = new DataString[3]; //shared | s_1 s_2 //0:1.0:0.5 | a_1 b_1 c_1 //| a_2 b_2 c_2 //| a_3 b_3 c_3 //| b_1 c_1 d_1 //0:0.0:0.5 | b_2 c_2 d_2 //| a_1 b_1 c_1 //| a_3 b_3 c_3 sampleData[0] = new DataString { Line = "shared | s_1 s_2", Shared = new[] { "s_1", "s_2" }, ActionDependentFeatures = new[] { new DataStringADF { Line = "0:1.0:0.5 | a_1 b_1 c_1", Features = new[] { "a_1", "b_1", "c_1" }, }, new DataStringADF { Line = "| a_2 b_2 c_2", Features = new [] { "a_2","b_2","c_2" } }, new DataStringADF { Line = "| a_3 b_3 c_3", Features = new [] { "a_3","b_3","c_3" } }, }, SelectedActionIndex = 0, Label = new ContextualBanditLabel { Cost = 1f, Probability = .5f } }; sampleData[1] = new DataString { Line = string.Empty, ActionDependentFeatures = new[] { new DataStringADF { Line = "| b_1 c_1 d_1", Features = new [] { "b_1","c_1","d_1" } }, new DataStringADF { Line = "0:0.0:0.5 | b_2 c_2 d_2", Features = new [] { "b_2", "c_2", "d_2" } }, }, SelectedActionIndex = 1, Label = new ContextualBanditLabel { Cost = 0f, Probability = .5f } }; sampleData[2] = new DataString { Line = string.Empty, ActionDependentFeatures = new[] { new DataStringADF { Line = "| a_1 b_1 c_1 ", Features = new [] { "a_1","b_1","c_1" } }, new DataStringADF { Line = "| a_3 b_3 c_3", Features = new [] { "a_3","b_3","c_3" } } } }; return sampleData; } private static DataString[] CreateStringCbAdfData(int numSamples, int randomSeed = 0) { var random = new Random(randomSeed); var sampleData = new DataString[numSamples]; for (int i = 0; i < numSamples; i++) { int numActions = random.Next(2, 5); int[] fIndex = Enumerable.Range(1, numActions).OrderBy(ind => random.Next()).Take(numActions).ToArray(); var features = new string[numActions][]; for (int j = 0; j < numActions; j++) { features[j] = new string[] { "a_" + fIndex[j], "b_" + fIndex[j], "c_" + fIndex[j], "d_" + fIndex[j] }; } var adf = new DataStringADF[numActions]; for (int j = 0; j < numActions; j++) { adf[j] = new DataStringADF { Features = features[j] }; } sampleData[i] = new DataString { ActionDependentFeatures = adf, SelectedActionIndex = random.Next(-1, numActions), Label = new ContextualBanditLabel { Cost = (float)random.NextDouble(), Probability = (float)random.NextDouble() } }; } return sampleData; } private DataFloat[] CreateFloatCbAdfData(int numSamples, int randomSeed = 0) { var random = new Random(randomSeed); var sampleData = new DataFloat[numSamples]; for (int i = 0; i < numSamples; i++) { int numActions = random.Next(2, 5); int[] fIndex = Enumerable.Range(1, numActions).OrderBy(ind => random.Next()).Take(numActions).ToArray(); var features = new float[numActions][]; for (int j = 0; j < numActions; j++) { features[j] = new float[] { (fIndex[j] + 0) / (float)numActions, (fIndex[j] + 1) / (float)numActions, (fIndex[j] + 2) / (float)numActions, (fIndex[j] + 3) / (float)numActions }; } var adf = new DataFloatADF[numActions]; for (int j = 0; j < numActions; j++) { adf[j] = new DataFloatADF { Features = features[j] }; } sampleData[i] = new DataFloat { ActionDependentFeatures = adf, SelectedActionIndex = random.Next(-1, numActions), Label = new ContextualBanditLabel { Cost = (float)random.NextDouble(), Probability = (float)random.NextDouble() } }; } return sampleData; } public class DataString { public string Line { get; set; } [Feature] public string[] Shared { get; set; } public IReadOnlyList ActionDependentFeatures { get; set; } public int SelectedActionIndex { get; set; } public ILabel Label { get; set; } } public class DataFloat { [Feature] public string[] Shared { get; set; } public IReadOnlyList ActionDependentFeatures { get; set; } public int SelectedActionIndex { get; set; } public ILabel Label { get; set; } } public class DataStringADF { public string Line { get; set; } [Feature] public string[] Features { get; set; } public override string ToString() { return string.Join(" ", this.Features); } } public class DataFloatADF { [Feature] public float[] Features { get; set; } public override string ToString() { return string.Join(" ", this.Features); } } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestConfidence.cs000066400000000000000000000047401323713302100230150ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Text; using System.Threading.Tasks; using VW; using VW.Labels; using VW.Serializer.Attributes; namespace cs_unittest { [TestClass] public class TestConfidenceClass { public class Data { [Feature] public double Value { get; set; } } [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestConfidence() { var rnd = new Random(42); using (var vw = new VowpalWabbit("--confidence -f model.conf --ngram 1 --bit_precision 8 --random_seed 123 --passes 2 -k -c model.conf.cache")) { for (int i = 0; i < 100; i++) { if (i % 2 == 0) vw.Learn(new Data { Value = rnd.NextDouble() }, new SimpleLabel { Label = 1 }); else vw.Learn(new Data { Value = rnd.NextDouble() + 3 }, new SimpleLabel { Label = -1 }); } vw.Native.RunMultiPass(); var pred = vw.Predict(new Data { Value = 4.5 }, VowpalWabbitPredictionType.ScalarConfidence); Assert.AreEqual(-1f, pred.Value); Assert.IsTrue(pred.Confidence > 5); } Assert.IsTrue(File.Exists("model.conf.cache.cache")); File.Delete("model.conf.cache.cache"); Assert.IsFalse(File.Exists("model.conf.cache.cache")); } [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestConfidenceWithStringLines() { var rnd = new Random(42); using (var vw = new VowpalWabbit("--confidence -f model.conf --ngram 1 --bit_precision 18 --random_seed 123 --passes 2 -k -c model.conf.cache")) { for (int i = 0; i < 100; i++) { if (i % 2 == 0) vw.Learn(string.Format("{0} | :{1}", 1, rnd.NextDouble())); else vw.Learn(string.Format("{0} | :{1}", -1, rnd.NextDouble() + 3)); } vw.Native.RunMultiPass(); var pred = vw.Predict(string.Format(" | :{0}", 4.5), VowpalWabbitPredictionType.ScalarConfidence); Assert.AreEqual(-1f, pred.Value); Assert.IsTrue(pred.Confidence > 5); } } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestConfigInspector.cs000066400000000000000000000071061323713302100240530ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections.Generic; using System.Diagnostics; using System.IO; using System.Linq; using System.Text; using System.Threading.Tasks; using VW; using VW.Labels; using VW.Serializer; namespace cs_unittest { [TestClass] public class TestConfigInspector { [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestConfigParsing() { var str = @" f1 ns1.f2(FeatureGroup = 'a') ns1.f3(Namespace = ""a"",Enumerize=true, FeatureGroup = 'g' ) ns1.f4(Enumerize=true , FeatureGroup ='f', Order=3,AddAnchor=true) ns1.ns2.f5 "; var schema = ConfigInspector.CreateSchema(typeof(ConfigSample), str, msg => Assert.Fail(msg)); using (var vw = new VowpalWabbitExampleValidator(new VowpalWabbitSettings { Schema = schema })) { vw.Validate("| f1:1 f5:5 |a abc |ga f33 |f f44 ", new ConfigSample { f1 = 1, ns1 = new ConfigSampleNamespace { f2 = "abc", f3 = 3, f4 = 4, ns2 = new ConfigSampleNamespaceSub { f5 = 5 } } }); } } [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestConfigADFParsing() { var schemaShared = ConfigInspector.CreateSchema(typeof(ConfigShared), "f1", msg => Assert.Fail(msg)); var schemaADF = ConfigInspector.CreateSchema(typeof(ConfigADF), "f2(Enumerize=true)", msg => Assert.Fail(msg)); using (var vw = new VowpalWabbit(new VowpalWabbitSettings("--cb_adf") { Schema = schemaShared, ActionDependentSchema = schemaADF })) using (var vwNative = new VowpalWabbit("--cb_adf")) { vw.Learn( new ConfigShared { f1 = 2, ignore_me = 3 }, new[] { new ConfigADF { f2 = 3 }, new ConfigADF { f2 = 4 }, }, 0, new ContextualBanditLabel { Action = 0, Cost = 1, Probability = .5f }); vwNative.Learn( new[] { "shared | f1:2", "0:1:.5 | f23", " | f24" }); vw.Native.SaveModel("config-actual.model"); vwNative.SaveModel("config-expected.model"); } var actual = File.ReadAllBytes("config-actual.model"); var expected = File.ReadAllBytes("config-expected.model"); CollectionAssert.AreEqual(expected, actual); } } public class ConfigSample { public int f1 { get; set; } public ConfigSampleNamespace ns1 { get; set; } } public class ConfigSampleNamespace { public string f2 { get; set; } public int f3 { get; set; } public int f4 { get; set; } public ConfigSampleNamespaceSub ns2 { get; set; } } public class ConfigSampleNamespaceSub { public int f5 { get; set; } } public class ConfigShared { public int f1 { get; set; } public int ignore_me { get; set; } } public class ConfigADF { public int f2 { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestConsole.cs000066400000000000000000000025651323713302100223650ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Text; using System.Threading.Tasks; using VW; namespace cs_unittest { [TestClass] public class TestConsoleClass : TestBase { [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestConsole() { var arrayModelPath = Path.GetTempFileName(); var newlineModelPath = Path.GetTempFileName(); var nativeModelPath = Path.GetTempFileName(); // Note: deployment item is not working on build server cs_vw.Program.Main(new[] { @"..\cs\unittest\json\test_array.json", "-f", arrayModelPath }); cs_vw.Program.Main(new[] { @"..\cs\unittest\json\test_newline.json", "-f", newlineModelPath }); // compare model using (var vw = new VowpalWabbit("-f " + nativeModelPath)) { vw.Learn("1 | f:1"); vw.Learn("0 | f:2"); } var arrayModel = File.ReadAllBytes(arrayModelPath); var newlineModel = File.ReadAllBytes(newlineModelPath); var nativeModel = File.ReadAllBytes(nativeModelPath); CollectionAssert.AreEqual(nativeModel, arrayModel); CollectionAssert.AreEqual(newlineModel, arrayModel); } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestDynamic.cs000066400000000000000000000066141323713302100223460ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Text; using System.Threading.Tasks; using VW; using VW.Labels; using VW.Serializer; namespace cs_unittest { [TestClass] public class TestDynamicClass { [TestMethod] [Ignore] [TestCategory("Vowpal Wabbit")] public void TestDynamic() { // TODO: look into friend assemblies and how to figure if one is a friend using (var vw = new VowpalWabbit("--cb_adf --rank_all")) using (var vwDynamic = new VowpalWabbitDynamic(new VowpalWabbitSettings("--cb_adf --rank_all") { TypeInspector = JsonTypeInspector.Default })) { var expected = vw.Learn(new[] { "| q:1", "2:-3:0.9 | q:2", "| q:3" }, VowpalWabbitPredictionType.ActionProbabilities); var actual = vwDynamic.Learn( new { _multi = new[] { new { q = 1 }, new { q = 2 }, new { q = 3 } } }, VowpalWabbitPredictionType.ActionScore, new ContextualBanditLabel(0, -3, 0.9f), 1); AssertAreEqual(expected, actual); expected = vw.Learn(new[] { "| q:1", "2:-5:0.9 | q:2", "| q:3" }, VowpalWabbitPredictionType.ActionProbabilities); actual = vwDynamic.Learn( new { _multi = new[] { new { q = 1 }, new { q = 2 }, new { q = 3 } } }, VowpalWabbitPredictionType.ActionScore, new ContextualBanditLabel(0, -5, 0.9f), 1); AssertAreEqual(expected, actual); expected = vw.Learn(new[] { "| q:1", "| q:2", "3:-2:0.8 | q:3" }, VowpalWabbitPredictionType.ActionProbabilities); actual = vwDynamic.Learn( new { _multi = new[] { new { q = 1 }, new { q = 2 }, new { q = 3 } }, _labelIndex = 2, _label_Action = 3, _label_Cost = -2, _label_Probability = 0.8 }, VowpalWabbitPredictionType.ActionScore); AssertAreEqual(expected, actual); } } private void AssertAreEqual(ActionScore[] expected, ActionScore[] actual) { Assert.AreEqual(expected.Length, actual.Length); for (int i = 0; i < expected.Length; i++) { Assert.AreEqual(expected[i].Action, actual[i].Action); Assert.AreEqual(expected[i].Score, actual[i].Score, 0.0001); } CollectionAssert.AreEqual( expected: Enumerable.Range(0, expected.Length).Select(i => (uint)i).ToList(), actual: actual.Select(a => a.Action).OrderBy(a => a).ToList()); } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestErrorListener.cs000066400000000000000000000010241323713302100235470ustar00rootroot00000000000000using Antlr4.Runtime; using Microsoft.VisualStudio.TestTools.UnitTesting; namespace cs_unittest { class TestErrorListener : IAntlrErrorListener { public void SyntaxError(IRecognizer recognizer, IToken offendingSymbol, int line, int charPositionInLine, string msg, RecognitionException e) { Assert.Fail("SyntaxError: {0} at line {1} character {2}: {3}", offendingSymbol, line, charPositionInLine, msg); } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestExampleCache.cs000066400000000000000000000103561323713302100232770ustar00rootroot00000000000000using System; using System.Collections.Generic; using System.IO; using Microsoft.VisualStudio.TestTools.UnitTesting; using VW; using VW.Labels; using VW.Serializer; using VW.Serializer.Attributes; namespace cs_unittest { [TestClass] public class TestExampleCacheCases : TestBase { #if DEBUG [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestExampleCacheForLearning() { try { using (var vw = new VowpalWabbit(new VowpalWabbitSettings { EnableExampleCaching = true })) { vw.Learn(new CachedData(), new SimpleLabel()); } Assert.Fail("Expect NotSupportedException"); } catch (NotSupportedException) { } } #else [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestExampleCacheForLearning() { try { using (var vw = new VowpalWabbit(new VowpalWabbitSettings { EnableExampleCaching = true })) { vw.Learn(new CachedData(), new SimpleLabel()); } Assert.Fail("Expect NullReferenceException"); } catch (NullReferenceException) { } } #endif [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestExampleCacheDisabledForLearning() { using (var vw = new VowpalWabbit(new VowpalWabbitSettings { EnableExampleCaching = false })) { vw.Learn(new CachedData(), new SimpleLabel()); } } [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestExampleCache() { var random = new Random(123); var examples = new List(); for (int i = 0; i < 1000; i++) { examples.Add(new CachedData { Label = new SimpleLabel { Label = 1 }, Feature = random.NextDouble() }); var cachedData = new CachedData { Label = new SimpleLabel { Label = 2 }, Feature = 10 + random.NextDouble() }; examples.Add(cachedData); examples.Add(cachedData); } using (var vw = new VowpalWabbit(new VowpalWabbitSettings("-k -c --passes 10") { EnableExampleCaching = false })) { foreach (var example in examples) { var pred = vw.Learn(example, example.Label, VowpalWabbitPredictionType.Scalar); //Console.WriteLine($"feature {example.Label.Label} <- {example.Feature}"); //Console.WriteLine($" pred {pred}"); } vw.Native.RunMultiPass(); vw.Native.SaveModel("models/model1"); } using (var vwModel = new VowpalWabbitModel(new VowpalWabbitSettings("-t") { ModelStream = File.OpenRead("models/model1") })) using (var vwCached = new VowpalWabbit(new VowpalWabbitSettings { Model = vwModel, EnableExampleCaching = true, MaxExampleCacheSize = 5 })) using (var vw = new VowpalWabbit(new VowpalWabbitSettings { Model = vwModel, EnableExampleCaching = false })) { foreach (var example in examples) { var cachedPrediction = vwCached.Predict(example, VowpalWabbitPredictionType.Scalar); var prediction = vw.Predict(example, VowpalWabbitPredictionType.Scalar); Assert.AreEqual(prediction, cachedPrediction); //Console.WriteLine($"{example.Label.Label} to {prediction} to {cachedPrediction} {example.Feature}"); Assert.AreEqual(example.Label.Label, Math.Round(prediction)); } } } } [Cacheable] public class CachedData { [Feature] public double Feature { get; set; } public SimpleLabel Label { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestExpansion.cs000066400000000000000000000020651323713302100227220ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using VW; using VW.Serializer.Attributes; namespace cs_unittest { [TestClass] public class TestExpansionClass { [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] [TestCategory("Vowpal Wabbit")] public void TestExpansion() { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| 3:.1 4:.2 5:.3", new ExpansionContext() { Features = new[] { .1f, .2f, .3f }, Offset = 3 }); } } } public class ExpansionContext { public float[] Features { get; set; } public int Offset { get; set; } [Feature] public IEnumerable ExpandedFeatures { get { return Enumerable.Repeat(0f, this.Offset) .Concat(Features); } } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestFeatureExtraction.cs000066400000000000000000000105071323713302100244120ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using VW; using VW.Serializer; using VW.Serializer.Attributes; namespace cs_unittest { [TestClass] public class TestFeatureExtractionClass { public class Features { [Feature] public float F1 { get; set; } [Feature(FeatureGroup = 'l')] public string Location { get; set; } } [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestFeatureExtraction() { using (var vw = new VowpalWabbit("--noconstant")) using (var serializer = vw.Serializer.Create(vw.Native)) using (var example = serializer.Serialize(new Features { F1 = 3.2f, Location = "New York" })) { var singleExample = example as VowpalWabbitSingleLineExampleCollection; Assert.IsNotNull(singleExample); foreach (var ns in singleExample.Example) { Console.WriteLine(ns.Index); foreach (var feature in ns) { Console.WriteLine("{0}:{1}", feature.FeatureIndex, feature.X); } } var namespaces = singleExample.Example.ToArray(); Assert.AreEqual((byte)' ', namespaces[0].Index); CollectionAssert.AreEqual( new[] { new VowpalWabbitFeature(singleExample.Example, 3.2f, 610696), }, namespaces[0].ToArray()); Assert.AreEqual((byte)'l', namespaces[1].Index); CollectionAssert.AreEqual( new[] { new VowpalWabbitFeature(singleExample.Example, 1, 414696), new VowpalWabbitFeature(singleExample.Example, 1, 380324), }, namespaces[1].ToArray()); } } [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestJsonFeatureExtraction() { string json = "{\"ns1\":{\"location\":\"New York\", \"f2\":3.4}}"; using (var vw = new VowpalWabbit("-b 3 --noconstant")) using (var serializer = new VowpalWabbitJsonSerializer(vw)) using (var result = serializer.ParseAndCreate(json)) { var singleExample = result as VowpalWabbitSingleLineExampleCollection; Assert.IsNotNull(singleExample); if (singleExample != null) { foreach (var ns in singleExample.Example) { Console.WriteLine(ns.Index); foreach (var feature in ns) { Console.WriteLine("{0}:{1}", feature.FeatureIndex, feature.X); } } var ns1 = singleExample.Example.ToArray(); Assert.AreEqual(1, ns1.Length); Assert.AreEqual((byte)'n', ns1[0].Index); CollectionAssert.AreEqual( new[] { new VowpalWabbitFeature(singleExample.Example, 1, 12), new VowpalWabbitFeature(singleExample.Example, 3.4f, 28) }, ns1[0].ToArray()); } // for documentation purpose only var multiExample = result as VowpalWabbitMultiLineExampleCollection; Assert.IsNull(multiExample); if (multiExample != null) { foreach (var example in multiExample.Examples) { foreach (var ns in example) { Console.WriteLine(ns.Index); foreach (var feature in ns) { Console.WriteLine("{0}:{1}", feature.FeatureIndex, feature.X); } } } } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestJson.cs000066400000000000000000002430641323713302100216750ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using Newtonsoft.Json; using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Text; using System.Threading.Tasks; using VW; using VW.Labels; using VW.Serializer; using VW.Serializer.Attributes; using VW.Serializer.Intermediate; namespace cs_unittest { [TestClass] public class TestJsonClass { [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJson() { using (var validator = new VowpalWabbitExampleJsonValidator()) { validator.Validate("|a foo:1", "{\"a\":{\"foo\":1}}"); validator.Validate("|a foo:2.3", "{\"a\":{\"foo\":2.3}}"); validator.Validate("|a foo:2.3 bar", "{\"a\":{\"foo\":2.3, \"bar\":true}}"); validator.Validate("|a foo:1 |bcd Age25_old", "{\"a\":{\"foo\":1},\"bcd\":{\"Age\":\"25 old\"}}"); validator.Validate("|a x{abc}", "{\"a\":{\"x\":\"{abc}\"}}"); validator.Validate("|a x{abc}", "{\"a\":{\"x\":\"{abc}\",\"y\":null}}"); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonAux() { using (var validator = new VowpalWabbitExampleJsonValidator()) { validator.Validate("|a foo:1", "{\"a\":{\"foo\":1},\"_aux\":5}"); validator.Validate("|a foo:1", "{\"a\":{\"foo\":1},\"_aux\":\"\"}"); validator.Validate("|a foo:1", "{\"a\":{\"foo\":1},\"_aux\":{\"abc\":{\"def\":3}}}"); validator.Validate("|a foo:1", "{\"a\":{\"foo\":1},\"_aux\":[1,2,[3,4],2]}"); validator.Validate("|a foo:1", "{\"a\":{\"foo\":1},\"_aux\":[1,2,[3,[1],{\"ab,\":3}],2]}"); validator.Validate("|a foo:1 | b:1", "{\"a\":{\"foo\":1},\"_aux\":{\"a\":\"{\\\"} \"}, \"b\":1}"); } } private void AssertThrow(Action action, Type expectedException = null) { if (expectedException == null) expectedException = typeof(VowpalWabbitException); try { action(); Assert.Fail("Expected exception " + expectedException); } catch (Exception e) { Assert.IsInstanceOfType(e, expectedException); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonInvalid() { using (var vw = new VowpalWabbit("--json")) { AssertThrow(() => vw.ParseJson("{\"_label\":true,\"a\":{\"foo\":1}}")); AssertThrow(() => vw.ParseJson("{\"_labelfoo\":1,\"a\":{\"foo\":1}}")); AssertThrow(() => vw.ParseJson("{\"_label_foo\":1,\"a\":{\"foo\":1}}")); AssertThrow(() => vw.ParseJson("{\"_label\":{\"label\":{\"a\":1}},\"a\":{\"foo\":1}}")); } using (var vw = new VowpalWabbit("--cb_adf --json")) { AssertThrow(() => vw.ParseJson("{\"_label_Action\":1,\"_label_Cost\":-2,\"_label_Probability\":0.3,\"_multi\":[{\"foo\":1}],\"foo\":2,\"_labelIndex\":1}")); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonArray() { using (var validator = new VowpalWabbitExampleJsonValidator()) { validator.Validate("|a :1 :2.3 :4", "{\"a\":[1,2.3,4]}"); validator.Validate("|b :1 :2.3 :4", "{\"a\":{\"b\":[1,2.3,4]}}"); } using (var vw = new VowpalWabbit("--json")) { AssertThrow(() => vw.ParseJson("{\"a\":{\"b\":[1,[1,2],4]}}")); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonSimpleLabel() { using (var validator = new VowpalWabbitExampleJsonValidator()) { validator.Validate("1 |a foo:1", "{\"_label\":{\"Label\":1},\"a\":{\"foo\":1}}", VowpalWabbitLabelComparator.Simple); validator.Validate("1.2 |a foo:1", "{\"_label\":1.2,\"a\":{\"foo\":1}}", VowpalWabbitLabelComparator.Simple); validator.Validate("1.2 |a foo:1", "{\"_label\":1.2,\"a\":{\"foo\":1}}", VowpalWabbitLabelComparator.Simple); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonVWLabel() { using (var validator = new VowpalWabbitExampleJsonValidator()) { validator.Validate("1 |a foo:1", "{\"_label\":1,\"a\":{\"foo\":1}}", VowpalWabbitLabelComparator.Simple); validator.Validate("1 |a foo:1", "{\"_label\":\"1\",\"a\":{\"foo\":1}}", VowpalWabbitLabelComparator.Simple); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonSimpleLabelOverride() { using (var validator = new VowpalWabbitExampleJsonValidator()) { validator.Validate("2 |a foo:1", "{\"_label\":{\"Label\":1},\"a\":{\"foo\":1}}", VowpalWabbitLabelComparator.Simple, new SimpleLabel { Label = 2 }, enableNativeJsonValidation: false /* vw.Parse(json) doesn't support label overwrite */); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonContextualBanditLabel() { using (var validator = new VowpalWabbitExampleJsonValidator("--cb 2 --cb_type dr")) { validator.Validate("1:2:.5 |a foo:1", "{\"_label\":\"1:2:.5\",\"a\":{\"foo\":1}}", VowpalWabbitLabelComparator.ContextualBandit); validator.Validate("1:-2:.3 |a foo:1", "{\"_label\":{\"Action\":1,\"Cost\":-2,\"Probability\":0.3},\"a\":{\"foo\":1}}", VowpalWabbitLabelComparator.ContextualBandit); validator.Validate("1:-2:.3 |a foo:1", "{\"_label_Action\":1,\"_label_Cost\":-2,\"_label_Probability\":0.3,\"a\":{\"foo\":1}}", VowpalWabbitLabelComparator.ContextualBandit); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonADF() { using (var validator = new VowpalWabbitExampleJsonValidator("--cb_adf")) { validator.Validate(new[] { "shared | foo:2", "1:-2:.3 | foo:1" }, "{\"_label_Action\":1,\"_label_Cost\":-2,\"_label_Probability\":0.3,\"_multi\":[{\"foo\":1}],\"foo\":2,\"_labelIndex\":0}", VowpalWabbitLabelComparator.ContextualBandit, index: 1); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonToVWString() { var jsonContext = new JsonContext() { Label = new SimpleLabel { Label = 25 }, Ns1 = new Namespace1 { Foo = 1, Age = "25", DontConsider = "XXX", EscapeCharacterString = "a: a | a\ta", EscapeCharactersText = "b: b | b\tb" }, Ns2 = new Namespace2 { FeatureA = true }, Clicks = 5 }; var jsonContextString = JsonConvert.SerializeObject(jsonContext); using (var validator = new VowpalWabbitExampleJsonValidator(new VowpalWabbitSettings { Arguments = "--json", EnableStringExampleGeneration = true, EnableStringFloatCompact = true, EnableThreadSafeExamplePooling = true })) { validator.Validate("25 | Clicks:5 MoreClicks:0 |a Bar:1 Age25 EscapeCharacterStringa__a___a_a b_ b _ b b |b Marker", jsonContextString, VowpalWabbitLabelComparator.Simple); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonMultiline() { using (var validator = new VowpalWabbitExampleJsonValidator("--cb 2 --cb_type dr")) { validator.Validate(new[] { "shared | Age:25", " | w1 w2 |a x:1", " | w2 w3" }, "{\"Age\":25,\"_multi\":[{\"_text\":\"w1 w2\", \"a\":{\"x\":1}}, {\"_text\":\"w2 w3\"}]}"); validator.Validate(new[] { "shared | Age:25", " | w1 w2 |a x:1", "2:-1:.3 | w2 w3" }, "{\"Age\":25,\"_multi\":[{\"_text\":\"w1 w2\", \"a\":{\"x\":1}}, {\"_text\":\"w2 w3\",\"_label\":\"2:-1:.3\"}]}", VowpalWabbitLabelComparator.ContextualBandit); } using (var validator = new VowpalWabbitExampleJsonValidator( new VowpalWabbitSettings { Arguments = "--cb 2 --cb_type dr", PropertyConfiguration = new PropertyConfiguration { MultiProperty = "adf", TextProperty = "someText", LabelProperty = "theLabel", FeatureIgnorePrefix = "xxx" } })) { validator.Validate(new[] { "shared | Age:25", " | w1 w2 |a x:1", "2:-1:.3 | w2 w3" }, "{\"Age\":25,\"adf\":[{\"someText\":\"w1 w2\", \"a\":{\"x\":1}, \"xxxxIgnoreMe\":2}, {\"someText\":\"w2 w3\",\"theLabel\":\"2:-1:.3\"}]}", VowpalWabbitLabelComparator.ContextualBandit, enableNativeJsonValidation: false /* remapping of special properties is not supported in native JSON */); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonText() { using (var validator = new VowpalWabbitExampleJsonValidator("")) { validator.Validate("| a b c |a d e f", "{\"_text\":\"a b c\",\"a\":{\"_text\":\"d e f\"}}"); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonNumADFs() { using (var validator = new VowpalWabbitExampleJsonValidator("")) { Assert.AreEqual(2, VowpalWabbitJsonSerializer.GetNumberOfActionDependentExamples( "{\"_text\":\"a b c\",\"a\":{\"_text\":\"d e f\"},_multi:[{\"a\":1},{\"b\":2,\"c\":3}]}")); Assert.AreEqual(0, VowpalWabbitJsonSerializer.GetNumberOfActionDependentExamples( "{\"_text\":\"a b c\",\"a\":{\"_text\":\"d e f\"},_multi:[]}")); Assert.AreEqual(0, VowpalWabbitJsonSerializer.GetNumberOfActionDependentExamples( "{\"_text\":\"a b c\",\"a\":{\"_text\":\"d e f\"}}")); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonLabel() { using (var validator = new VowpalWabbitExampleJsonValidator("")) { validator.Validate("1 | a:2 ", "{\"a\":2,\"_label_Label\":1}"); validator.Validate("1:2:3 | a:2 ", "{\"a\":2,\"_label_Label\":1,\"_label_Initial\":2,\"_label_weight\":3}"); } using (var validator = new VowpalWabbitExampleJsonValidator(new VowpalWabbitSettings { Arguments = "--cb_adf", PropertyConfiguration = new PropertyConfiguration { MultiProperty = "adf", TextProperty = "someText", FeatureIgnorePrefix = "xxx" } })) { validator.Validate(new[] { "shared | Age:25", " | w1 w2 |a x:1", "0:-1:.3 | w2 w3" }, "{\"Age\":25,\"adf\":[{\"someText\":\"w1 w2\", \"a\":{\"x\":1}, \"xxxxIgnoreMe\":2}, {\"someText\":\"w2 w3\"}], \"_labelIndex\":1, \"_label_Cost\":-1, \"_label_Probability\":0.3}", VowpalWabbitLabelComparator.ContextualBandit, enableNativeJsonValidation: false); // all lower case (ASA issue) validator.Validate(new[] { " | w1 w2 |a x:1", "0:-1:.3 | w2 w3" }, "{\"adf\":[{\"someText\":\"w1 w2\", \"a\":{\"x\":1}, \"xxxxIgnoreMe\":2}, {\"someText\":\"w2 w3\"}], \"_labelindex\":1, \"_label_cost\":-1, \"_label_probability\":0.3}", VowpalWabbitLabelComparator.ContextualBandit, enableNativeJsonValidation: false); validator.Validate(new[] { "shared | Age:25", " | w1 w2 |a x:1", " | w2 w3" }, "{\"Age\":25,\"adf\":[{\"someText\":\"w1 w2\", \"a\":{\"x\":1}, \"xxxxIgnoreMe\":2}, {\"someText\":\"w2 w3\"}], \"_labelindex\":null}", VowpalWabbitLabelComparator.ContextualBandit, enableNativeJsonValidation: false); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonLabelExtraction() { using (var vw = new VowpalWabbit("--cb_adf --rank_all")) { using (var jsonSerializer = new VowpalWabbitJsonSerializer(vw)) { string eventId = null; jsonSerializer.RegisterExtension((state, property) => { Assert.AreEqual(property, "_eventid"); Assert.IsTrue(state.Reader.Read()); eventId = (string)state.Reader.Value; return true; }); jsonSerializer.Parse("{\"_eventid\":\"abc123\",\"a\":1,\"_label_cost\":-1,\"_label_probability\":0.3}"); Assert.AreEqual("abc123", eventId); using (var examples = jsonSerializer.CreateExamples()) { var single = examples as VowpalWabbitSingleLineExampleCollection; Assert.IsNotNull(single); var label = single.Example.Label as ContextualBanditLabel; Assert.IsNotNull(label); Assert.AreEqual(-1, label.Cost); Assert.AreEqual(0.3, label.Probability, 0.0001); } } using (var jsonSerializer = new VowpalWabbitJsonSerializer(vw)) { jsonSerializer.Parse("{\"_multi\":[{\"_text\":\"w1 w2\", \"a\":{\"x\":1}}, {\"_text\":\"w2 w3\"}], \"_labelindex\":1, \"_label_cost\":-1, \"_label_probability\":0.3}"); using (var examples = jsonSerializer.CreateExamples()) { var multi = examples as VowpalWabbitMultiLineExampleCollection; Assert.IsNotNull(multi); Assert.AreEqual(2, multi.Examples.Length); var label = multi.Examples[0].Label as ContextualBanditLabel; Assert.AreEqual(0, label.Cost); Assert.AreEqual(0, label.Probability); label = multi.Examples[1].Label as ContextualBanditLabel; Assert.IsNotNull(label); Assert.AreEqual(-1, label.Cost); Assert.AreEqual(0.3, label.Probability, 0.0001); } } } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonRedirection() { using (var validator = new VowpalWabbitExampleJsonValidator(new VowpalWabbitSettings("--cb_adf"))) { validator.Validate(new[] { "shared | Age:25", " | w1 w2 |a x:1", "0:-1:.3 | w2 w3" }, "{\"_ignoreMe\":5,\"_sub\":{\"Age\":25,\"_multi\":[{\"_text\":\"w1 w2\", \"a\":{\"x\":1}}, {\"_text\":\"w2 w3\"}]}, \"_labelIndex\":1, \"_label_Cost\":-1, \"_label_Probability\":0.3}", VowpalWabbitLabelComparator.ContextualBandit, extension: (state, property) => { if (!property.Equals("_sub")) return false; Assert.AreEqual(state.MultiIndex, -1); state.Parse(); return true; }); validator.Validate(new[] { "shared | Age:25", " | w1 w2 |a x:1", "0:-1:.3 | w2 w3" }, "{\"Age\":25,\"_multi\":[{\"_text\":\"w1 w2\", \"a\":{\"x\":1}}, {\"_text\":\"w2 w3\", \"_tag\":\"2\"}], \"_labelIndex\":1, \"_label_Cost\":-1, \"_label_Probability\":0.3}", VowpalWabbitLabelComparator.ContextualBandit, extension: (state, property) => { if (!property.Equals("_tag")) return false; var tag = state.Reader.ReadAsString(); Assert.AreEqual(1, state.MultiIndex); Assert.AreEqual("2", tag); return true; }); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestDecisionServiceJson() { using (var vw = new VowpalWabbit("--cb_adf")) { var json = @"{""EventId"":""abc"",""a"":[1,2,3],""Version"":""1"",""c"":{""u"":{""loc"":""New York""},""_multi"":[{""x"":[{""x"":{""cat"":""1""}},null,{""y"":{""cat"":""3""}}]},{""x"":{""cat"":""2""}}]},""p"":[0.8,0.1,0.1]}"; var obj = JsonConvert.DeserializeObject(json); var bytes = Encoding.UTF8.GetBytes(json); VowpalWabbitDecisionServiceInteractionHeader header; List examples = null; try { examples = vw.ParseDecisionServiceJson(bytes, 0, bytes.Length, copyJson: false, header: out header); Assert.AreEqual("abc", header.EventId); CollectionAssert.AreEqual(new[] { 1, 2, 3 }, header.Actions, "Actions mismatch"); CollectionAssert.AreEqual(new[] { .8f, .1f, .1f }, header.Probabilities, "Probabilities mismatch"); Assert.AreEqual(0, header.ProbabilityOfDrop); using (var validator = new VowpalWabbitExampleJsonValidator(new VowpalWabbitSettings("--cb_adf"))) { var expected = new[] { "shared |u locNew_York", " |x cat1 |y cat3", " |x cat2" }; validator.Validate(expected, examples); } } finally { if (examples != null) foreach (var ex in examples) if (ex != null) ex.Dispose(); } } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestDecisionServiceJson2() { var json = "{\"Version\":\"2\",\"EventId\":\"73369b13ec98433096a1496d27da0bfd\",\"a\":[9,11,13,6,4,5,12,1,2,10,8,3,7],\"c\":{\"_synthetic\":false,\"User\":{\"_age\":0},\"Geo\":{\"country\":\"United States\",\"_countrycf\":\"8\",\"state\":\"New Jersey\",\"city\":\"Somerdale\",\"_citycf\":\"5\",\"dma\":\"504\"},\"MRefer\":{\"referer\":\"http://www.complex.com/\"},\"OUserAgent\":{\"_ua\":\"Mozilla/5.0 (iPhone; CPU iPhone OS 10_3_2 like Mac OS X) AppleWebKit/603.2.4 (KHTML, like Gecko) Version/10.0 Mobile/14F89 Safari/602.1\",\"_DeviceBrand\":\"Apple\",\"_DeviceFamily\":\"iPhone\",\"_DeviceIsSpider\":false,\"_DeviceModel\":\"iPhone\",\"_OSFamily\":\"iOS\",\"_OSMajor\":\"10\",\"_OSPatch\":\"2\",\"DeviceType\":\"Mobile\"},\"_multi\":[{\"_tag\":\"cmplx$http://www.complex.com/sports/2017/06/floyd-mayweater-conor-mcgregor-may-be-set\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/sports/2017/06/floyd-mayweater-conor-mcgregor-may-be-set\"},\"j\":[{\"_title\":\"The Floyd Mayweather vs. Conor McGregor Fight Date Has Finally Been Announced\"},{\"RVisionTags\":{\"person\":0.999368966,\"man\":0.998108864,\"wearing\":0.9368642,\"hat\":0.928866565,\"indoor\":0.893332958,\"close\":0.201101109},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0148094837,\"racyScore\":0.0144806523},\"_expires\":\"2017-06-17T21:42:30.3186957Z\"},{\"Emotion0\":{\"anger\":1.81591489E-07,\"contempt\":9.946987E-06,\"disgust\":6.11135547E-05,\"fear\":4.565633E-12,\"happiness\":0.999928534,\"neutral\":2.28114558E-07,\"sadness\":3.07409E-09,\"surprise\":1.46155665E-08},\"_expires\":\"2017-06-17T21:42:28.8496462Z\"},{\"Tags\":{\"Floyd Mayweather Jr.\":0.982,\"Conor McGregor\":0.938,\"Complex\":0.334,\"Twitter Inc.\":0.997,\"Dan Mullane\":0.006,\"Mixed martial arts\":1,\"Net Controls\":0.281,\"Boxing\":1,\"Ontario\":1,\"Dana White\":0.972,\"Las Vegas Valley\":0.995,\"Nevada Athletic Commission\":0.024,\"Mayweather Promotions\":0.118,\"MGM Grand Garden Arena\":0.997,\"Fighting game\":0.641,\"Nevada\":1,\"Coming out\":0.076},\"_expires\":\"2017-06-17T21:42:29.8271823Z\"},{\"XSentiment\":2.93618323E-05,\"_expires\":\"2017-06-17T21:42:29.1777863Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/music/2017/06/young-thug-beautiful-thugger-girls-violent-trailer-has-people-upset\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/music/2017/06/young-thug-beautiful-thugger-girls-violent-trailer-has-people-upset\"},\"j\":[{\"_title\":\"Why Young Thug's Violent Trailer for 'Beautiful Thugger Girls' Has People Upset\"},{\"RVisionTags\":{\"person\":0.9621564,\"indoor\":0.93759197},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0590519942,\"racyScore\":0.0740057454},\"_expires\":\"2017-06-17T17:27:41.0750729Z\"},{\"Emotion0\":{\"anger\":0.000195411936,\"contempt\":0.0007970728,\"disgust\":7.29157255E-05,\"fear\":0.000106336483,\"happiness\":0.000127831052,\"neutral\":0.9209777,\"sadness\":0.0775285438,\"surprise\":0.000194183245},\"_expires\":\"2017-06-17T17:27:33.4982953Z\"},{\"Tags\":{\"Young Thug\":0.445,\"Beautiful\":0.005,\"Twitter Inc.\":1,\"Drake\":0.968,\"Instagram\":0.995,\"Breezy\":0.014,\"June 13\":0.008,\"Cover art\":0.003,\"Album\":0.99,\"Surface\":0.012,\"Prince Michael Jackson II\":1},\"_expires\":\"2017-06-17T17:27:33.4670277Z\"},{\"XSentiment\":2.46002031E-07,\"_expires\":\"2017-06-17T17:27:34.0139318Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/sports/2017/06/did-we-just-witness-peak-lebron\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/sports/2017/06/did-we-just-witness-peak-lebron\"},\"j\":[{\"_title\":\"Did We Just Witness Peak LeBron?\"},{\"RVisionTags\":{\"person\":0.9887383,\"indoor\":0.8961688},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0181511845,\"racyScore\":0.0380923077},\"_expires\":\"2017-06-17T14:46:55.4499648Z\"},{\"Emotion0\":{\"anger\":0.0007875018,\"contempt\":0.000600368367,\"disgust\":0.00288117747,\"fear\":0.00168624625,\"happiness\":4.59727671E-05,\"neutral\":0.377137423,\"sadness\":0.00264490047,\"surprise\":0.6142164},\"_expires\":\"2017-06-17T14:46:49.7072998Z\"},{\"Tags\":{\"LeBron James\":1,\"Complex\":0.08,\"Broadcasting of sports events\":0.12,\"Twitter Inc.\":0.999,\"USA Today\":0.994,\"Kyle Broflovski\":0.023,\"Superman\":0.694,\"Cleveland Cavaliers\":1,\"UNK NBA\":1,\"Bill Russell NBA Finals Most Valuable Player Award\":1,\"Golden State Warriors\":1,\"Michael Jordan\":1,\"Kobe Bryant\":1,\"Time\":0.865,\"Magic Johnson\":0.999},\"_expires\":\"2017-06-17T14:46:49.4416286Z\"},{\"XSentiment\":0.9999974,\"_expires\":\"2017-06-17T14:46:49.7541575Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/sports/2017/06/kevin-durant-made-the-difference\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/sports/2017/06/kevin-durant-made-the-difference\"},\"j\":[{\"_title\":\"Golden State (Probably) Would Have Blown Another Lead Without KD\"},{\"RVisionTags\":{\"person\":0.999950767,\"player\":0.984833837,\"sport\":0.9816471,\"athletic game\":0.9691899,\"basketball\":0.7260069,\"hand\":0.463383943,\"crowd\":0.3354485},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0127091147,\"racyScore\":0.01585682},\"_expires\":\"2017-06-17T14:44:47.7452593Z\"},{\"Emotion0\":{\"anger\":0.9830929,\"contempt\":6.38814454E-05,\"disgust\":0.0142428661,\"fear\":4.3443215E-05,\"happiness\":0.00127731345,\"neutral\":0.0003242454,\"sadness\":0.0006526729,\"surprise\":0.0003026811},\"_expires\":\"2017-06-17T14:44:47.1857099Z\"},{\"Tags\":{\"Golden State Warriors\":1,\"Lead guitar\":0.046,\"Kevin Durant\":1,\"Twitter Inc.\":0.985,\"Martinez\":0.015,\"Splash Brothers\":1,\"Cleveland Cavaliers\":1,\"Monday Night Football\":0.398,\"Richard Jefferson\":0.95,\"Kevin Love\":0.997,\"McDonald's All-American Game\":0.859,\"University of Texas at Austin College of Fine Arts\":0.999,\"Naismith College Player of the Year\":0.117,\"UNK NBA\":1,\"NBA Most Valuable Player Award\":1,\"Olympic Games\":0.858,\"NBA All-Star Game\":1,\"Champion\":0.12,\"Bill Russell NBA Finals Most Valuable Player Award\":1,\"Stephen Curry\":0.989,\"Draymond Green\":0.674,\"Seat Pleasant\":0.122},\"_expires\":\"2017-06-17T14:44:46.9264793Z\"},{\"XSentiment\":1,\"_expires\":\"2017-06-17T14:44:47.4639789Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/music/2017/06/2-chainz-dna-freestyle-kendrick-lamar\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/music/2017/06/2-chainz-dna-freestyle-kendrick-lamar\"},\"j\":[{\"_title\":\"Watch 2 Chainz Flex on Kendrick Lamar's \\\"DNA\\\" Beat in New Freestyle\"},{\"RVisionTags\":{\"person\":0.9840661,\"dressed\":0.304036647},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0285988934,\"racyScore\":0.0257010516},\"_expires\":\"2017-06-17T19:30:46.5524171Z\"},{\"_expires\":\"2017-06-17T19:30:46.2086571Z\"},{\"Tags\":{\"2 Chainz\":1,\"Kendrick Lamar\":1,\"DNA\":0.053,\"Hip hop production\":0.015,\"Freestyle rap\":0.934,\"Philadelphia\":0.193,\"Twitter Inc.\":1,\"Subscription business model\":0.011,\"Complex\":1,\"Los Angeles\":1,\"Georgia\":0.894,\"Trap\":0.579,\"Top Dawg Entertainment\":0.872,\"Virtual reality\":0.011,\"Travis Scott\":0.011,\"Collaboration\":0.004,\"Nicki Minaj\":0.999,\"Remy Ma\":0.99,\"Papoose\":0.892,\"Sampling\":0.017,\"Hip hop music\":1,\"Tha Carter V\":1,\"Everyday (ASAP Rocky song)\":0.003},\"_expires\":\"2017-06-17T19:30:46.1314041Z\"},{\"XSentiment\":1,\"_expires\":\"2017-06-17T19:30:46.7944713Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/sports/2017/06/floyd-mayweather-viral-challenge-backfires-reacts\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/sports/2017/06/floyd-mayweather-viral-challenge-backfires-reacts\"},\"j\":[{\"_title\":\"Floyd Mayweather Attempted to Start His Own Viral Challenge and It Hilariously Backfired\"},{\"RVisionTags\":{\"person\":0.999924064,\"man\":0.9552084,\"crowd\":0.01707872},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0114686647,\"racyScore\":0.0159121435},\"_expires\":\"2017-06-17T21:13:01.1925796Z\"},{\"Emotion0\":{\"anger\":0.00127403683,\"contempt\":0.0222781487,\"disgust\":0.0002176114,\"fear\":9.893078E-06,\"happiness\":0.07823167,\"neutral\":0.897131145,\"sadness\":0.00048493495,\"surprise\":0.0003725856},\"_expires\":\"2017-06-17T21:12:59.3669891Z\"},{\"Tags\":{\"Floyd Mayweather Jr.\":0.192,\"Philadelphia\":0.65,\"Twitter Inc.\":1,\"USA Today\":0.979,\"Sport\":0.85,\"Conor McGregor\":0.016,\"June 14\":0.009,\"Troy\":0.005,\"Honda Civic\":0.004,\"Bank account\":0.025,\"NASCAR on TNT\":0.099,\"Boxing\":1},\"_expires\":\"2017-06-17T21:12:59.3201526Z\"},{\"XSentiment\":0.999998,\"_expires\":\"2017-06-17T21:13:01.7647699Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/sports/2017/06/kevin-durant-discusses-random-text-he-received-from-obama-after-winning-nba-finals\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/sports/2017/06/kevin-durant-discusses-random-text-he-received-from-obama-after-winning-nba-finals\"},\"j\":[{\"_title\":\"Kevin Durant Discusses 'Random' Text He Received From Obama After Winning NBA Finals\"},{\"RVisionTags\":{\"person\":0.9939494,\"outdoor\":0.9162231,\"male\":0.242890328},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0208446719,\"racyScore\":0.0304868072},\"_expires\":\"2017-06-17T18:28:04.5675692Z\"},{\"Emotion0\":{\"anger\":2.35385942E-05,\"contempt\":3.294205E-06,\"disgust\":1.38025935E-05,\"fear\":7.8729E-06,\"happiness\":0.9997165,\"neutral\":0.0001607666,\"sadness\":1.25915176E-05,\"surprise\":6.161377E-05},\"Emotion1\":{\"anger\":0.00371822272,\"contempt\":0.000460597221,\"disgust\":0.000157746123,\"fear\":0.000275517668,\"happiness\":0.0403934456,\"neutral\":0.86978966,\"sadness\":0.08499769,\"surprise\":0.000207107121},\"_expires\":\"2017-06-17T18:28:03.3944386Z\"},{\"Tags\":{\"Kevin Durant\":1,\"Random House\":0.069,\"Barack Obama\":1,\"UNK NBA\":1,\"Twitter Inc.\":1,\"USA Today\":0.999,\"Sports journalism\":0.01,\"Cary\":0.025,\"The NBA Finals\":0.017,\"Monday Night Football\":0.943,\"Golden State Warriors\":1,\"Bill Simmons\":0.866,\"Podcast\":0.918,\"Oracle Arena\":0.028,\"Bill Russell NBA Finals Most Valuable Player Award\":1,\"June 13\":0.034,\"Cleveland Cavaliers\":1,\"LeBron James\":1,\"Kyrie Irving\":1,\"Allen Iverson\":0.999,\"Rihanna\":0.63,\"The League\":0.166,\"Stay\":0.858,\"Singing\":0.024,\"President of the United States\":1},\"_expires\":\"2017-06-17T18:28:03.8094333Z\"},{\"XSentiment\":0.9999663,\"_expires\":\"2017-06-17T18:28:03.9031587Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/pop-culture/2017/06/tj-miller-hbo-special-interview\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/pop-culture/2017/06/tj-miller-hbo-special-interview\"},\"j\":[{\"_title\":\"T.J. Miller's Done With 'Silicon Valley,' But His Career's Just Getting Started\"},{\"RVisionTags\":{\"person\":0.999560535,\"man\":0.9939658,\"suit\":0.950625,\"outdoor\":0.9169477,\"wearing\":0.7708228,\"jacket\":0.528340757,\"coat\":0.490623325,\"dark\":0.32463637,\"male\":0.212570518,\"microphone\":0.148984566,\"crowd\":0.0156043554},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0102083739,\"racyScore\":0.0126593616},\"_expires\":\"2017-06-17T18:59:23.2248419Z\"},{\"Emotion0\":{\"anger\":1.0740634E-05,\"contempt\":1.42603E-05,\"disgust\":5.26589356E-05,\"fear\":1.20671484E-06,\"happiness\":0.996862,\"neutral\":0.00302408962,\"sadness\":5.683868E-06,\"surprise\":2.93453577E-05},\"_expires\":\"2017-06-17T18:59:22.2148365Z\"},{\"Tags\":{\"T. J. Miller\":0.999,\"Silicon Valley\":0.999,\"Whitney\":0.264,\"Twitter Inc.\":1,\"HBO\":1,\"Complex\":0.032,\"The Gorburger Show\":0.062,\"Funny or Die\":0.914,\"Comedy Central\":1,\"Japan\":1,\"Ridiculousness\":0.004,\"Deadpool\":0.117,\"Cloverfield\":0.921,\"Cannes Film Festival\":0.987,\"Energizer Bunny\":0.007,\"Amy Schumer\":0.021,\"Pete Holmes\":0.153,\"Peter Boyle\":0.003,\"Downtown Los Angeles\":0.843,\"Supervillain\":0.038,\"San Francisco\":0.809,\"Jesus Christ\":0.902,\"Kong: Skull Island\":0.751,\"Jordan Vogt-Roberts\":0.979,\"Usher\":0.325,\"Flea\":0.536,\"Henry Rollins\":0.096,\"Federal government of the United States\":0.485,\"Mike Judge\":0.974,\"Uber\":0.404,\"Chelsea Handler\":0.364},\"_expires\":\"2017-06-17T18:59:22.7021704Z\"},{\"_expires\":\"2017-06-17T18:59:22.7489821Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/life/2017/06/iphone-8-edge-to-edge-screen\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/life/2017/06/iphone-8-edge-to-edge-screen\"},\"j\":[{\"_title\":\"Newly-Leaked Pictures Show You What iPhone 8 Screen Might Look Like\"},{\"RVisionTags\":{\"iPod\":0.802692235,\"electronics\":0.7313406},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0145561891,\"racyScore\":0.0151244327},\"_expires\":\"2017-06-17T23:14:19.5133399Z\"},{\"_expires\":\"2017-06-17T23:14:18.3277963Z\"},{\"Tags\":{\"iOS\":0.492,\"Complex\":0.102,\"Twitter Inc.\":1,\"Imgur\":0.996,\"Apple Inc.\":1,\"Reddit\":0.57,\"Check It Out\":0.003,\"China\":0.813,\"iPhone\":1},\"_expires\":\"2017-06-17T23:14:18.0245382Z\"},{\"XSentiment\":0.9994883,\"_expires\":\"2017-06-17T23:14:18.7364306Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/sports/2017/06/ryan-destiny-get-sweaty\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/sports/2017/06/ryan-destiny-get-sweaty\"},\"j\":[{\"_title\":\"Ryan Destiny Talks About Starring in Hit TV Series 'Star' on Get Sweaty With Emily Oberg\"},{\"RVisionTags\":{\"person\":0.998844266,\"boxing\":0.573501348},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0180808976,\"racyScore\":0.07170816},\"_expires\":\"2017-06-17T19:55:45.6659091Z\"},{\"Emotion0\":{\"anger\":0.0428811572,\"contempt\":0.0120455148,\"disgust\":0.00588818826,\"fear\":0.0031699785,\"happiness\":0.0325962044,\"neutral\":0.839067638,\"sadness\":0.0560076348,\"surprise\":0.00834368449},\"Emotion1\":{\"anger\":9.522394E-08,\"contempt\":3.42922242E-08,\"disgust\":3.1531672E-06,\"fear\":4.24115054E-09,\"happiness\":0.999994,\"neutral\":2.3093894E-06,\"sadness\":1.65964408E-07,\"surprise\":2.82413E-07},\"_expires\":\"2017-06-17T19:55:44.6508371Z\"},{\"Tags\":{\"Destiny\":0.299,\"HiT TV\":0.008,\"Complex\":0.953,\"Twitter Inc.\":1,\"New York City\":1,\"Robert E. Lee\":0.004,\"Fox Broadcasting Company\":1,\"Naomi Campbell\":0.091,\"Sy Kravitz\":0.731,\"Queen Latifah\":0.876},\"_expires\":\"2017-06-17T19:55:44.2710467Z\"},{\"XSentiment\":0.113136955,\"_expires\":\"2017-06-17T19:55:45.0269667Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/music/2017/06/2-chainz-dna-freestyle-kendrick-lamar\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/music/2017/06/2-chainz-dna-freestyle-kendrick-lamar\"},\"j\":[{\"_title\":\"Watch 2 Chainz Flex on Kendrick Lamar's \\\"DNA\\\" Beat in New Freestyle\"}]},{\"_tag\":\"cmplx$http://www.complex.com/music/2017/06/everyday-struggle-ep39-kehlani-tinashe\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/music/2017/06/everyday-struggle-ep39-kehlani-tinashe\"},\"j\":[{\"_title\":\"Joe Budden and DJ Akademiks Discuss Tinashe Controversy and Kehlani Cussing Out Heckler on 'Everyday Struggle'\"},{\"RVisionTags\":{\"abstract\":0.5319324},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.09698458,\"racyScore\":0.08712957},\"_expires\":\"2017-06-17T17:56:47.5354099Z\"},{\"Emotion0\":{\"anger\":0.0121765705,\"contempt\":0.0152475182,\"disgust\":0.0186378732,\"fear\":0.00241010683,\"happiness\":0.202662408,\"neutral\":0.6867056,\"sadness\":0.0549196824,\"surprise\":0.00724024652},\"Emotion1\":{\"anger\":0.005693211,\"contempt\":0.0003093396,\"disgust\":0.000264122762,\"fear\":0.000122387908,\"happiness\":0.000162528377,\"neutral\":0.9901545,\"sadness\":0.00110488839,\"surprise\":0.00218904181},\"_expires\":\"2017-06-17T17:56:46.5115225Z\"},{\"Tags\":{\"Joe Budden\":0.972,\"Disc jockey\":0.998,\"Tinashe\":0.485,\"Kehlani\":0.011,\"Profanity\":0.006,\"Heckler\":0.003,\"Complex\":0.992,\"Twitter Inc.\":0.959,\"XXL\":0.906,\"Kyrie Irving\":0.277,\"LeBron James\":0.998,\"Michael Jordan\":0.989,\"Floyd Mayweather Jr.\":0.031,\"Conor McGregor\":0.276},\"_expires\":\"2017-06-17T17:56:46.0896119Z\"},{\"XSentiment\":0.005011654,\"_expires\":\"2017-06-17T17:56:46.6990201Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/sports/2017/06/lonzo-ball-interview\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/sports/2017/06/lonzo-ball-interview\"},\"j\":[{\"_title\":\"Lonzo Ball Finally Told Us How He Really Feels About LaVar's Media Antics\"},{\"RVisionTags\":{\"person\":0.9992661,\"sport\":0.9894762,\"athletic game\":0.9814596,\"basketball\":0.9540427,\"player\":0.897939742,\"crowd\":0.529209554,\"watching\":0.450753957},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0182933789,\"racyScore\":0.0168850645},\"_expires\":\"2017-06-17T14:11:20.108671Z\"},{\"Emotion0\":{\"anger\":6.044781E-05,\"contempt\":2.85142833E-05,\"disgust\":4.95703644E-05,\"fear\":0.008861069,\"happiness\":2.874653E-05,\"neutral\":0.8961255,\"sadness\":0.00528799742,\"surprise\":0.08955817},\"_expires\":\"2017-06-17T14:11:19.6899531Z\"},{\"Tags\":{\"Los Angeles Angels of Anaheim\":0.515,\"Songwriter\":0.33,\"Twitter Inc.\":1,\"USA Today\":1,\"Broadcasting of sports events\":0.038,\"Chino Hills\":0.002,\"UCLA Bruins men's basketball\":0.6,\"NCAA Men's Division I Basketball Championship\":0.897,\"Sweet\":0.019,\"Todd Marinovich\":0.938,\"Marv Albert\":0.055,\"UNK NBA\":1,\"Fox Broadcasting Company\":0.933,\"Jayson Tatum\":0.004,\"ZO2\":0.008,\"Los Angeles Lakers\":1,\"Lamar Odom\":0.986,\"Lamar Cardinals Men's Basketball\":0.004,\"Magic Johnson\":0.995,\"Jason Kidd\":1,\"LeBron James\":1,\"James Harden\":0.99,\"Adidas\":0.811,\"Puerto Rico\":0.012,\"Stephen Curry\":0.981,\"Michael Jordan\":1,\"Shaquille O'Neal\":1},\"_expires\":\"2017-06-17T14:11:19.6587216Z\"},{\"XSentiment\":1,\"_expires\":\"2017-06-17T14:11:20.0892306Z\"}]}]},\"p\":[0.8153846,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154],\"VWState\":{\"m\":\"decc63fa2c284ec9887ee0572ea16d17/7860031216114d8bb718c40abc801bf4\"}}"; using (var vw = new VowpalWabbit("--cb_adf")) { var obj = JsonConvert.DeserializeObject(json); var bytes = new byte[Encoding.UTF8.GetMaxByteCount(json.Length) + 1]; var bytes2 = new byte[Encoding.UTF8.GetMaxByteCount(json.Length) + 1]; var byteLen = Encoding.UTF8.GetBytes(json, 0, json.Length, bytes, 0) + 1;// trailing \0 Array.Copy(bytes, bytes2, bytes.Length); VowpalWabbitDecisionServiceInteractionHeader header; List examples = null; try { examples = vw.ParseDecisionServiceJson(bytes, 0, byteLen, copyJson: false, header: out header); Assert.AreEqual("73369b13ec98433096a1496d27da0bfd", header.EventId); CollectionAssert.AreEqual(new[] { 9, 11, 13, 6, 4, 5, 12, 1, 2, 10, 8, 3, 7 }, header.Actions, "Actions mismatch"); CollectionAssert.AreEqual(new[] { 0.8153846f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f }, header.Probabilities, "Probabilities mismatch"); Assert.AreEqual(0, header.ProbabilityOfDrop); Assert.AreEqual(14, examples.Count); // check if copyJson: false was actually used CollectionAssert.AreNotEqual(bytes2, bytes); } finally { if (examples != null) foreach (var ex in examples) if (ex != null) ex.Dispose(); } } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestDecisionServiceJson_CopyJson() { var json = "{\"Version\":\"2\",\"EventId\":\"73369b13ec98433096a1496d27da0bfd\",\"a\":[9,11,13,6,4,5,12,1,2,10,8,3,7],\"c\":{\"_synthetic\":false,\"User\":{\"_age\":0},\"Geo\":{\"country\":\"United States\",\"_countrycf\":\"8\",\"state\":\"New Jersey\",\"city\":\"Somerdale\",\"_citycf\":\"5\",\"dma\":\"504\"},\"MRefer\":{\"referer\":\"http://www.complex.com/\"},\"OUserAgent\":{\"_ua\":\"Mozilla/5.0 (iPhone; CPU iPhone OS 10_3_2 like Mac OS X) AppleWebKit/603.2.4 (KHTML, like Gecko) Version/10.0 Mobile/14F89 Safari/602.1\",\"_DeviceBrand\":\"Apple\",\"_DeviceFamily\":\"iPhone\",\"_DeviceIsSpider\":false,\"_DeviceModel\":\"iPhone\",\"_OSFamily\":\"iOS\",\"_OSMajor\":\"10\",\"_OSPatch\":\"2\",\"DeviceType\":\"Mobile\"},\"_multi\":[{\"_tag\":\"cmplx$http://www.complex.com/sports/2017/06/floyd-mayweater-conor-mcgregor-may-be-set\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/sports/2017/06/floyd-mayweater-conor-mcgregor-may-be-set\"},\"j\":[{\"_title\":\"The Floyd Mayweather vs. Conor McGregor Fight Date Has Finally Been Announced\"},{\"RVisionTags\":{\"person\":0.999368966,\"man\":0.998108864,\"wearing\":0.9368642,\"hat\":0.928866565,\"indoor\":0.893332958,\"close\":0.201101109},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0148094837,\"racyScore\":0.0144806523},\"_expires\":\"2017-06-17T21:42:30.3186957Z\"},{\"Emotion0\":{\"anger\":1.81591489E-07,\"contempt\":9.946987E-06,\"disgust\":6.11135547E-05,\"fear\":4.565633E-12,\"happiness\":0.999928534,\"neutral\":2.28114558E-07,\"sadness\":3.07409E-09,\"surprise\":1.46155665E-08},\"_expires\":\"2017-06-17T21:42:28.8496462Z\"},{\"Tags\":{\"Floyd Mayweather Jr.\":0.982,\"Conor McGregor\":0.938,\"Complex\":0.334,\"Twitter Inc.\":0.997,\"Dan Mullane\":0.006,\"Mixed martial arts\":1,\"Net Controls\":0.281,\"Boxing\":1,\"Ontario\":1,\"Dana White\":0.972,\"Las Vegas Valley\":0.995,\"Nevada Athletic Commission\":0.024,\"Mayweather Promotions\":0.118,\"MGM Grand Garden Arena\":0.997,\"Fighting game\":0.641,\"Nevada\":1,\"Coming out\":0.076},\"_expires\":\"2017-06-17T21:42:29.8271823Z\"},{\"XSentiment\":2.93618323E-05,\"_expires\":\"2017-06-17T21:42:29.1777863Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/music/2017/06/young-thug-beautiful-thugger-girls-violent-trailer-has-people-upset\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/music/2017/06/young-thug-beautiful-thugger-girls-violent-trailer-has-people-upset\"},\"j\":[{\"_title\":\"Why Young Thug's Violent Trailer for 'Beautiful Thugger Girls' Has People Upset\"},{\"RVisionTags\":{\"person\":0.9621564,\"indoor\":0.93759197},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0590519942,\"racyScore\":0.0740057454},\"_expires\":\"2017-06-17T17:27:41.0750729Z\"},{\"Emotion0\":{\"anger\":0.000195411936,\"contempt\":0.0007970728,\"disgust\":7.29157255E-05,\"fear\":0.000106336483,\"happiness\":0.000127831052,\"neutral\":0.9209777,\"sadness\":0.0775285438,\"surprise\":0.000194183245},\"_expires\":\"2017-06-17T17:27:33.4982953Z\"},{\"Tags\":{\"Young Thug\":0.445,\"Beautiful\":0.005,\"Twitter Inc.\":1,\"Drake\":0.968,\"Instagram\":0.995,\"Breezy\":0.014,\"June 13\":0.008,\"Cover art\":0.003,\"Album\":0.99,\"Surface\":0.012,\"Prince Michael Jackson II\":1},\"_expires\":\"2017-06-17T17:27:33.4670277Z\"},{\"XSentiment\":2.46002031E-07,\"_expires\":\"2017-06-17T17:27:34.0139318Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/sports/2017/06/did-we-just-witness-peak-lebron\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/sports/2017/06/did-we-just-witness-peak-lebron\"},\"j\":[{\"_title\":\"Did We Just Witness Peak LeBron?\"},{\"RVisionTags\":{\"person\":0.9887383,\"indoor\":0.8961688},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0181511845,\"racyScore\":0.0380923077},\"_expires\":\"2017-06-17T14:46:55.4499648Z\"},{\"Emotion0\":{\"anger\":0.0007875018,\"contempt\":0.000600368367,\"disgust\":0.00288117747,\"fear\":0.00168624625,\"happiness\":4.59727671E-05,\"neutral\":0.377137423,\"sadness\":0.00264490047,\"surprise\":0.6142164},\"_expires\":\"2017-06-17T14:46:49.7072998Z\"},{\"Tags\":{\"LeBron James\":1,\"Complex\":0.08,\"Broadcasting of sports events\":0.12,\"Twitter Inc.\":0.999,\"USA Today\":0.994,\"Kyle Broflovski\":0.023,\"Superman\":0.694,\"Cleveland Cavaliers\":1,\"UNK NBA\":1,\"Bill Russell NBA Finals Most Valuable Player Award\":1,\"Golden State Warriors\":1,\"Michael Jordan\":1,\"Kobe Bryant\":1,\"Time\":0.865,\"Magic Johnson\":0.999},\"_expires\":\"2017-06-17T14:46:49.4416286Z\"},{\"XSentiment\":0.9999974,\"_expires\":\"2017-06-17T14:46:49.7541575Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/sports/2017/06/kevin-durant-made-the-difference\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/sports/2017/06/kevin-durant-made-the-difference\"},\"j\":[{\"_title\":\"Golden State (Probably) Would Have Blown Another Lead Without KD\"},{\"RVisionTags\":{\"person\":0.999950767,\"player\":0.984833837,\"sport\":0.9816471,\"athletic game\":0.9691899,\"basketball\":0.7260069,\"hand\":0.463383943,\"crowd\":0.3354485},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0127091147,\"racyScore\":0.01585682},\"_expires\":\"2017-06-17T14:44:47.7452593Z\"},{\"Emotion0\":{\"anger\":0.9830929,\"contempt\":6.38814454E-05,\"disgust\":0.0142428661,\"fear\":4.3443215E-05,\"happiness\":0.00127731345,\"neutral\":0.0003242454,\"sadness\":0.0006526729,\"surprise\":0.0003026811},\"_expires\":\"2017-06-17T14:44:47.1857099Z\"},{\"Tags\":{\"Golden State Warriors\":1,\"Lead guitar\":0.046,\"Kevin Durant\":1,\"Twitter Inc.\":0.985,\"Martinez\":0.015,\"Splash Brothers\":1,\"Cleveland Cavaliers\":1,\"Monday Night Football\":0.398,\"Richard Jefferson\":0.95,\"Kevin Love\":0.997,\"McDonald's All-American Game\":0.859,\"University of Texas at Austin College of Fine Arts\":0.999,\"Naismith College Player of the Year\":0.117,\"UNK NBA\":1,\"NBA Most Valuable Player Award\":1,\"Olympic Games\":0.858,\"NBA All-Star Game\":1,\"Champion\":0.12,\"Bill Russell NBA Finals Most Valuable Player Award\":1,\"Stephen Curry\":0.989,\"Draymond Green\":0.674,\"Seat Pleasant\":0.122},\"_expires\":\"2017-06-17T14:44:46.9264793Z\"},{\"XSentiment\":1,\"_expires\":\"2017-06-17T14:44:47.4639789Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/music/2017/06/2-chainz-dna-freestyle-kendrick-lamar\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/music/2017/06/2-chainz-dna-freestyle-kendrick-lamar\"},\"j\":[{\"_title\":\"Watch 2 Chainz Flex on Kendrick Lamar's \\\"DNA\\\" Beat in New Freestyle\"},{\"RVisionTags\":{\"person\":0.9840661,\"dressed\":0.304036647},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0285988934,\"racyScore\":0.0257010516},\"_expires\":\"2017-06-17T19:30:46.5524171Z\"},{\"_expires\":\"2017-06-17T19:30:46.2086571Z\"},{\"Tags\":{\"2 Chainz\":1,\"Kendrick Lamar\":1,\"DNA\":0.053,\"Hip hop production\":0.015,\"Freestyle rap\":0.934,\"Philadelphia\":0.193,\"Twitter Inc.\":1,\"Subscription business model\":0.011,\"Complex\":1,\"Los Angeles\":1,\"Georgia\":0.894,\"Trap\":0.579,\"Top Dawg Entertainment\":0.872,\"Virtual reality\":0.011,\"Travis Scott\":0.011,\"Collaboration\":0.004,\"Nicki Minaj\":0.999,\"Remy Ma\":0.99,\"Papoose\":0.892,\"Sampling\":0.017,\"Hip hop music\":1,\"Tha Carter V\":1,\"Everyday (ASAP Rocky song)\":0.003},\"_expires\":\"2017-06-17T19:30:46.1314041Z\"},{\"XSentiment\":1,\"_expires\":\"2017-06-17T19:30:46.7944713Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/sports/2017/06/floyd-mayweather-viral-challenge-backfires-reacts\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/sports/2017/06/floyd-mayweather-viral-challenge-backfires-reacts\"},\"j\":[{\"_title\":\"Floyd Mayweather Attempted to Start His Own Viral Challenge and It Hilariously Backfired\"},{\"RVisionTags\":{\"person\":0.999924064,\"man\":0.9552084,\"crowd\":0.01707872},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0114686647,\"racyScore\":0.0159121435},\"_expires\":\"2017-06-17T21:13:01.1925796Z\"},{\"Emotion0\":{\"anger\":0.00127403683,\"contempt\":0.0222781487,\"disgust\":0.0002176114,\"fear\":9.893078E-06,\"happiness\":0.07823167,\"neutral\":0.897131145,\"sadness\":0.00048493495,\"surprise\":0.0003725856},\"_expires\":\"2017-06-17T21:12:59.3669891Z\"},{\"Tags\":{\"Floyd Mayweather Jr.\":0.192,\"Philadelphia\":0.65,\"Twitter Inc.\":1,\"USA Today\":0.979,\"Sport\":0.85,\"Conor McGregor\":0.016,\"June 14\":0.009,\"Troy\":0.005,\"Honda Civic\":0.004,\"Bank account\":0.025,\"NASCAR on TNT\":0.099,\"Boxing\":1},\"_expires\":\"2017-06-17T21:12:59.3201526Z\"},{\"XSentiment\":0.999998,\"_expires\":\"2017-06-17T21:13:01.7647699Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/sports/2017/06/kevin-durant-discusses-random-text-he-received-from-obama-after-winning-nba-finals\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/sports/2017/06/kevin-durant-discusses-random-text-he-received-from-obama-after-winning-nba-finals\"},\"j\":[{\"_title\":\"Kevin Durant Discusses 'Random' Text He Received From Obama After Winning NBA Finals\"},{\"RVisionTags\":{\"person\":0.9939494,\"outdoor\":0.9162231,\"male\":0.242890328},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0208446719,\"racyScore\":0.0304868072},\"_expires\":\"2017-06-17T18:28:04.5675692Z\"},{\"Emotion0\":{\"anger\":2.35385942E-05,\"contempt\":3.294205E-06,\"disgust\":1.38025935E-05,\"fear\":7.8729E-06,\"happiness\":0.9997165,\"neutral\":0.0001607666,\"sadness\":1.25915176E-05,\"surprise\":6.161377E-05},\"Emotion1\":{\"anger\":0.00371822272,\"contempt\":0.000460597221,\"disgust\":0.000157746123,\"fear\":0.000275517668,\"happiness\":0.0403934456,\"neutral\":0.86978966,\"sadness\":0.08499769,\"surprise\":0.000207107121},\"_expires\":\"2017-06-17T18:28:03.3944386Z\"},{\"Tags\":{\"Kevin Durant\":1,\"Random House\":0.069,\"Barack Obama\":1,\"UNK NBA\":1,\"Twitter Inc.\":1,\"USA Today\":0.999,\"Sports journalism\":0.01,\"Cary\":0.025,\"The NBA Finals\":0.017,\"Monday Night Football\":0.943,\"Golden State Warriors\":1,\"Bill Simmons\":0.866,\"Podcast\":0.918,\"Oracle Arena\":0.028,\"Bill Russell NBA Finals Most Valuable Player Award\":1,\"June 13\":0.034,\"Cleveland Cavaliers\":1,\"LeBron James\":1,\"Kyrie Irving\":1,\"Allen Iverson\":0.999,\"Rihanna\":0.63,\"The League\":0.166,\"Stay\":0.858,\"Singing\":0.024,\"President of the United States\":1},\"_expires\":\"2017-06-17T18:28:03.8094333Z\"},{\"XSentiment\":0.9999663,\"_expires\":\"2017-06-17T18:28:03.9031587Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/pop-culture/2017/06/tj-miller-hbo-special-interview\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/pop-culture/2017/06/tj-miller-hbo-special-interview\"},\"j\":[{\"_title\":\"T.J. Miller's Done With 'Silicon Valley,' But His Career's Just Getting Started\"},{\"RVisionTags\":{\"person\":0.999560535,\"man\":0.9939658,\"suit\":0.950625,\"outdoor\":0.9169477,\"wearing\":0.7708228,\"jacket\":0.528340757,\"coat\":0.490623325,\"dark\":0.32463637,\"male\":0.212570518,\"microphone\":0.148984566,\"crowd\":0.0156043554},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0102083739,\"racyScore\":0.0126593616},\"_expires\":\"2017-06-17T18:59:23.2248419Z\"},{\"Emotion0\":{\"anger\":1.0740634E-05,\"contempt\":1.42603E-05,\"disgust\":5.26589356E-05,\"fear\":1.20671484E-06,\"happiness\":0.996862,\"neutral\":0.00302408962,\"sadness\":5.683868E-06,\"surprise\":2.93453577E-05},\"_expires\":\"2017-06-17T18:59:22.2148365Z\"},{\"Tags\":{\"T. J. Miller\":0.999,\"Silicon Valley\":0.999,\"Whitney\":0.264,\"Twitter Inc.\":1,\"HBO\":1,\"Complex\":0.032,\"The Gorburger Show\":0.062,\"Funny or Die\":0.914,\"Comedy Central\":1,\"Japan\":1,\"Ridiculousness\":0.004,\"Deadpool\":0.117,\"Cloverfield\":0.921,\"Cannes Film Festival\":0.987,\"Energizer Bunny\":0.007,\"Amy Schumer\":0.021,\"Pete Holmes\":0.153,\"Peter Boyle\":0.003,\"Downtown Los Angeles\":0.843,\"Supervillain\":0.038,\"San Francisco\":0.809,\"Jesus Christ\":0.902,\"Kong: Skull Island\":0.751,\"Jordan Vogt-Roberts\":0.979,\"Usher\":0.325,\"Flea\":0.536,\"Henry Rollins\":0.096,\"Federal government of the United States\":0.485,\"Mike Judge\":0.974,\"Uber\":0.404,\"Chelsea Handler\":0.364},\"_expires\":\"2017-06-17T18:59:22.7021704Z\"},{\"_expires\":\"2017-06-17T18:59:22.7489821Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/life/2017/06/iphone-8-edge-to-edge-screen\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/life/2017/06/iphone-8-edge-to-edge-screen\"},\"j\":[{\"_title\":\"Newly-Leaked Pictures Show You What iPhone 8 Screen Might Look Like\"},{\"RVisionTags\":{\"iPod\":0.802692235,\"electronics\":0.7313406},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0145561891,\"racyScore\":0.0151244327},\"_expires\":\"2017-06-17T23:14:19.5133399Z\"},{\"_expires\":\"2017-06-17T23:14:18.3277963Z\"},{\"Tags\":{\"iOS\":0.492,\"Complex\":0.102,\"Twitter Inc.\":1,\"Imgur\":0.996,\"Apple Inc.\":1,\"Reddit\":0.57,\"Check It Out\":0.003,\"China\":0.813,\"iPhone\":1},\"_expires\":\"2017-06-17T23:14:18.0245382Z\"},{\"XSentiment\":0.9994883,\"_expires\":\"2017-06-17T23:14:18.7364306Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/sports/2017/06/ryan-destiny-get-sweaty\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/sports/2017/06/ryan-destiny-get-sweaty\"},\"j\":[{\"_title\":\"Ryan Destiny Talks About Starring in Hit TV Series 'Star' on Get Sweaty With Emily Oberg\"},{\"RVisionTags\":{\"person\":0.998844266,\"boxing\":0.573501348},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0180808976,\"racyScore\":0.07170816},\"_expires\":\"2017-06-17T19:55:45.6659091Z\"},{\"Emotion0\":{\"anger\":0.0428811572,\"contempt\":0.0120455148,\"disgust\":0.00588818826,\"fear\":0.0031699785,\"happiness\":0.0325962044,\"neutral\":0.839067638,\"sadness\":0.0560076348,\"surprise\":0.00834368449},\"Emotion1\":{\"anger\":9.522394E-08,\"contempt\":3.42922242E-08,\"disgust\":3.1531672E-06,\"fear\":4.24115054E-09,\"happiness\":0.999994,\"neutral\":2.3093894E-06,\"sadness\":1.65964408E-07,\"surprise\":2.82413E-07},\"_expires\":\"2017-06-17T19:55:44.6508371Z\"},{\"Tags\":{\"Destiny\":0.299,\"HiT TV\":0.008,\"Complex\":0.953,\"Twitter Inc.\":1,\"New York City\":1,\"Robert E. Lee\":0.004,\"Fox Broadcasting Company\":1,\"Naomi Campbell\":0.091,\"Sy Kravitz\":0.731,\"Queen Latifah\":0.876},\"_expires\":\"2017-06-17T19:55:44.2710467Z\"},{\"XSentiment\":0.113136955,\"_expires\":\"2017-06-17T19:55:45.0269667Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/music/2017/06/2-chainz-dna-freestyle-kendrick-lamar\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/music/2017/06/2-chainz-dna-freestyle-kendrick-lamar\"},\"j\":[{\"_title\":\"Watch 2 Chainz Flex on Kendrick Lamar's \\\"DNA\\\" Beat in New Freestyle\"}]},{\"_tag\":\"cmplx$http://www.complex.com/music/2017/06/everyday-struggle-ep39-kehlani-tinashe\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/music/2017/06/everyday-struggle-ep39-kehlani-tinashe\"},\"j\":[{\"_title\":\"Joe Budden and DJ Akademiks Discuss Tinashe Controversy and Kehlani Cussing Out Heckler on 'Everyday Struggle'\"},{\"RVisionTags\":{\"abstract\":0.5319324},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.09698458,\"racyScore\":0.08712957},\"_expires\":\"2017-06-17T17:56:47.5354099Z\"},{\"Emotion0\":{\"anger\":0.0121765705,\"contempt\":0.0152475182,\"disgust\":0.0186378732,\"fear\":0.00241010683,\"happiness\":0.202662408,\"neutral\":0.6867056,\"sadness\":0.0549196824,\"surprise\":0.00724024652},\"Emotion1\":{\"anger\":0.005693211,\"contempt\":0.0003093396,\"disgust\":0.000264122762,\"fear\":0.000122387908,\"happiness\":0.000162528377,\"neutral\":0.9901545,\"sadness\":0.00110488839,\"surprise\":0.00218904181},\"_expires\":\"2017-06-17T17:56:46.5115225Z\"},{\"Tags\":{\"Joe Budden\":0.972,\"Disc jockey\":0.998,\"Tinashe\":0.485,\"Kehlani\":0.011,\"Profanity\":0.006,\"Heckler\":0.003,\"Complex\":0.992,\"Twitter Inc.\":0.959,\"XXL\":0.906,\"Kyrie Irving\":0.277,\"LeBron James\":0.998,\"Michael Jordan\":0.989,\"Floyd Mayweather Jr.\":0.031,\"Conor McGregor\":0.276},\"_expires\":\"2017-06-17T17:56:46.0896119Z\"},{\"XSentiment\":0.005011654,\"_expires\":\"2017-06-17T17:56:46.6990201Z\"}]},{\"_tag\":\"cmplx$http://www.complex.com/sports/2017/06/lonzo-ball-interview\",\"i\":{\"constant\":1,\"id\":\"cmplx$http://www.complex.com/sports/2017/06/lonzo-ball-interview\"},\"j\":[{\"_title\":\"Lonzo Ball Finally Told Us How He Really Feels About LaVar's Media Antics\"},{\"RVisionTags\":{\"person\":0.9992661,\"sport\":0.9894762,\"athletic game\":0.9814596,\"basketball\":0.9540427,\"player\":0.897939742,\"crowd\":0.529209554,\"watching\":0.450753957},\"SVisionAdult\":{\"isAdultContent\":false,\"isRacyContent\":false,\"adultScore\":0.0182933789,\"racyScore\":0.0168850645},\"_expires\":\"2017-06-17T14:11:20.108671Z\"},{\"Emotion0\":{\"anger\":6.044781E-05,\"contempt\":2.85142833E-05,\"disgust\":4.95703644E-05,\"fear\":0.008861069,\"happiness\":2.874653E-05,\"neutral\":0.8961255,\"sadness\":0.00528799742,\"surprise\":0.08955817},\"_expires\":\"2017-06-17T14:11:19.6899531Z\"},{\"Tags\":{\"Los Angeles Angels of Anaheim\":0.515,\"Songwriter\":0.33,\"Twitter Inc.\":1,\"USA Today\":1,\"Broadcasting of sports events\":0.038,\"Chino Hills\":0.002,\"UCLA Bruins men's basketball\":0.6,\"NCAA Men's Division I Basketball Championship\":0.897,\"Sweet\":0.019,\"Todd Marinovich\":0.938,\"Marv Albert\":0.055,\"UNK NBA\":1,\"Fox Broadcasting Company\":0.933,\"Jayson Tatum\":0.004,\"ZO2\":0.008,\"Los Angeles Lakers\":1,\"Lamar Odom\":0.986,\"Lamar Cardinals Men's Basketball\":0.004,\"Magic Johnson\":0.995,\"Jason Kidd\":1,\"LeBron James\":1,\"James Harden\":0.99,\"Adidas\":0.811,\"Puerto Rico\":0.012,\"Stephen Curry\":0.981,\"Michael Jordan\":1,\"Shaquille O'Neal\":1},\"_expires\":\"2017-06-17T14:11:19.6587216Z\"},{\"XSentiment\":1,\"_expires\":\"2017-06-17T14:11:20.0892306Z\"}]}]},\"p\":[0.8153846,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154],\"VWState\":{\"m\":\"decc63fa2c284ec9887ee0572ea16d17/7860031216114d8bb718c40abc801bf4\"}}"; using (var vw = new VowpalWabbit("--cb_adf")) { var obj = JsonConvert.DeserializeObject(json); var bytes = new byte[Encoding.UTF8.GetMaxByteCount(json.Length) + 1]; var bytes2 = new byte[Encoding.UTF8.GetMaxByteCount(json.Length) + 1]; var byteLen = Encoding.UTF8.GetBytes(json, 0, json.Length, bytes, 0) + 1;// trailing \0 Array.Copy(bytes, bytes2, bytes.Length); VowpalWabbitDecisionServiceInteractionHeader header; List examples = null; try { examples = vw.ParseDecisionServiceJson(bytes, 0, byteLen, copyJson: true, header: out header); Assert.AreEqual("73369b13ec98433096a1496d27da0bfd", header.EventId); CollectionAssert.AreEqual(new[] { 9, 11, 13, 6, 4, 5, 12, 1, 2, 10, 8, 3, 7 }, header.Actions, "Actions mismatch"); CollectionAssert.AreEqual(new[] { 0.8153846f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f }, header.Probabilities, "Probabilities mismatch"); Assert.AreEqual(0, header.ProbabilityOfDrop); Assert.AreEqual(14, examples.Count); // check if copyJson: true was actually used CollectionAssert.AreEqual(bytes2, bytes); } finally { if (examples != null) foreach (var ex in examples) if (ex != null) ex.Dispose(); } } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestDecisionServiceJsonNull() { var json = @"{""Version"":""1"",""EventId"":""7cacacea2c6e49b5b922f6f517a325ed"",""a"":[9,4,13,10,8,5,2,3,12,11,7,6,1],""c"":{""_synthetic"":false,""User"":{""_age"":0},""Geo"":{""country"":""United States"",""_countrycf"":""8"",""state"":""Georgia"",""city"":""Stone Mountain"",""_citycf"":""5"",""dma"":""524""},""MRefer"":{""referer"":""http://www.complex.com/""},""OUserAgent"":{""_ua"":""Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/603.2.4 (KHTML, like Gecko) Version/10.1.1 Safari/603.2.4"",""_DeviceBrand"":"""",""_DeviceFamily"":""Other"",""_DeviceIsSpider"":false,""_DeviceModel"":"""",""_OSFamily"":""Mac OS X"",""_OSMajor"":""10"",""_OSPatch"":""5"",""DeviceType"":""Desktop""},""_multi"":[{""_tag"":""cmplx$http://www.complex.com/music/2017/06/prodigy-mobb-deep-once-in-a-generation-rapper"",""i"":{""constant"":1,""id"":""cmplx$http://www.complex.com/music/2017/06/prodigy-mobb-deep-once-in-a-generation-rapper""},""j"":[{""_title"":""Why Prodigy Was A Once-In-A-Generation Rapper""},{""RVisionTags"":{""person"":0.9913805,""hat"":0.6433856,""male"":0.153918922},""SVisionAdult"":{""isAdultContent"":false,""isRacyContent"":false,""adultScore"":0.0162594952,""racyScore"":0.0152094},""TVisionCelebrities"":{""Prodigy"":0.9999119},""_expires"":""2017-06-24T22:43:00.9241929Z""},{""Emotion0"":{""anger"":0.005261584,""contempt"":0.01940289,""disgust"":0.00146069494,""fear"":7.486289E-05,""happiness"":0.0102698216,""neutral"":0.9544214,""sadness"":0.00859182,""surprise"":0.00051694276},""_expires"":""2017-06-24T22:43:00.0008415Z""},{""Tags"":{""Roc Marciano"":0.015,""Mobb Deep"":1,""Prodigy"":1,""Havoc"":1,""A Tribe Called Quest"":0.969,""Q-Tip"":0.992,""The Infamous"":1,""The Crystals"":0.01,""Fifth Beatle"":0.099},""_expires"":""2017-06-24T22:43:00.1727355Z""},{""XSentiment"":3.01036973E-13,""_expires"":""2017-06-24T22:43:00.9398236Z""}]},{""_tag"":""cmplx$http://www.complex.com/pop-culture/2017/06/best-movies-2017"",""i"":{""constant"":1,""id"":""cmplx$http://www.complex.com/pop-culture/2017/06/best-movies-2017""},""j"":[{""_title"":""The Best Movies of 2017 (So Far)""},{""RVisionTags"":{""text"":0.9992092,""book"":0.997986436},""SVisionAdult"":{""isAdultContent"":false,""isRacyContent"":false,""adultScore"":0.0431842171,""racyScore"":0.0587232448},""_expires"":""2017-06-25T03:08:01.2111373Z""},{""Emotion0"":{""anger"":0.005728849,""contempt"":0.00117533945,""disgust"":8.215821E-05,""fear"":1.32827354E-05,""happiness"":0.000487715733,""neutral"":0.9911194,""sadness"":0.000120451317,""surprise"":0.00127282448},""_expires"":""2017-06-25T03:08:00.8636383Z""},null,{""XSentiment"":1,""_expires"":""2017-06-25T03:08:01.5112947Z""}]},{""_tag"":""cmplx$http://www.complex.com/music/2017/06/rihanna-responds-to-dm-from-fan-seeking-advice-on-getting-over-his-first-heartbreak"",""i"":{""constant"":1,""id"":""cmplx$http://www.complex.com/music/2017/06/rihanna-responds-to-dm-from-fan-seeking-advice-on-getting-over-his-first-heartbreak""},""j"":[{""_title"":""Rihanna Responds to DM From Fan Seeking Advice on Getting Over His First Heartbreak""},{""RVisionTags"":{""person"":0.99633044},""SVisionAdult"":{""isAdultContent"":false,""isRacyContent"":false,""adultScore"":0.013516671,""racyScore"":0.048148632},""_expires"":""2017-06-25T02:01:20.2363494Z""},{""Emotion0"":{""anger"":3.085194E-06,""contempt"":0.000411317043,""disgust"":1.026042E-05,""fear"":3.766979E-07,""happiness"":0.9783716,""neutral"":0.0211082119,""sadness"":5.66137569E-05,""surprise"":3.850023E-05},""_expires"":""2017-06-25T02:01:19.7050726Z""},{""Tags"":{""Rihanna"":1,""Martinez"":0.021,""Complex"":0.834,""Twitter Inc."":1,""If You"":0.003,""Grammy Awards"":1,""Mathematics"":0.999,""Malawi"":0.298,""Christine Teigen"":0.009,""Dave Chappelle"":0.047,""Presenter"":0.892,""Kendrick Lamar"":0.984,""Diamonds"":0.998},""_expires"":""2017-06-25T02:01:19.4155039Z""},{""XSentiment"":1,""_expires"":""2017-06-25T02:01:24.9467431Z""}]},{""_tag"":""cmplx$http://www.complex.com/music/2017/06/rihanna-responds-to-dm-from-fan-seeking-advice-on-getting-over-his-first-heartbreak"",""i"":{""constant"":1,""id"":""cmplx$http://www.complex.com/music/2017/06/rihanna-responds-to-dm-from-fan-seeking-advice-on-getting-over-his-first-heartbreak""},""j"":[{""_title"":""Rihanna Responds to DM From Fan Seeking Advice on Getting Over His First Heartbreak""}]},{""_tag"":""cmplx$http://www.complex.com/life/2017/06/guy-changes-from-shorts-to-dress-after-getting-sent-home-from-work-on-hot-day"",""i"":{""constant"":1,""id"":""cmplx$http://www.complex.com/life/2017/06/guy-changes-from-shorts-to-dress-after-getting-sent-home-from-work-on-hot-day""},""j"":[{""_title"":""Guy Sent Home by Boss for Wearing Shorts on a Hot Day, Returns to Work in Mom's Dress""},{""RVisionTags"":{""person"":0.999545038,""indoor"":0.998509467,""wall"":0.997952163},""SVisionAdult"":{""isAdultContent"":false,""isRacyContent"":false,""adultScore"":0.034187492,""racyScore"":0.0335518233},""_expires"":""2017-06-24T21:27:46.7135793Z""},{""Emotion0"":{""anger"":7.7880286E-05,""contempt"":0.00116215891,""disgust"":4.65850326E-06,""fear"":3.797253E-06,""happiness"":2.82076635E-05,""neutral"":0.986993253,""sadness"":0.0116343917,""surprise"":9.564323E-05},""_expires"":""2017-06-24T21:27:46.1197986Z""},{""Tags"":{""Boss Corporation"":0.007,""Shorts"":0.139,""Complex"":0.169,""Twitter Inc."":1,""English"":0.603,""The Daily Mirror"":0.008,""Oklahoma"":0.948,""High school"":0.999,""Lists of National Basketball Association players"":0.072,""UNK NBA"":0.529,""NBA dress code"":0.031,""Dress code"":0.113},""_expires"":""2017-06-24T21:27:46.0729078Z""},{""XSentiment"":0.9995655,""_expires"":""2017-06-24T21:27:46.7448359Z""}]},{""_tag"":""cmplx$http://www.complex.com/pop-culture/2017/06/tom-cruise-allegedly-balanced-bible-study-and-blow-jobs-away-from-risky-business-set"",""i"":{""constant"":1,""id"":""cmplx$http://www.complex.com/pop-culture/2017/06/tom-cruise-allegedly-balanced-bible-study-and-blow-jobs-away-from-risky-business-set""},""j"":[{""_title"":""Tom Cruise Was Allegedly Balancing Bible Study and Blow Jobs Away From the 'Risky Business' Set""},{""RVisionTags"":{""person"":0.9992724,""man"":0.9368908},""SVisionAdult"":{""isAdultContent"":false,""isRacyContent"":false,""adultScore"":0.009141326,""racyScore"":0.00951602},""TVisionCelebrities"":{""TOM CRUISE"":0.999997854},""_expires"":""2017-06-24T22:43:00.578981Z""},{""Emotion0"":{""anger"":1.77455458E-05,""contempt"":0.000173967157,""disgust"":0.000229260724,""fear"":3.15066657E-08,""happiness"":0.9786317,""neutral"":0.0209334176,""sadness"":5.861598E-07,""surprise"":1.33144977E-05},""_expires"":""2017-06-24T22:43:00.0793561Z""},{""Tags"":{""Connor Cruise"":1,""The Bible"":1,""Risky Business"":0.999,""Martinez"":0.006,""Complex"":0.01,""Twitter Inc."":1,""Sean Penn"":0.989,""Curtis Armstrong"":0.625,""The Hollywood Reporter"":0.203,""Louis Armstrong"":0.348,""Cruise ship"":1,""Chicago"":0.997,""Rebecca De Mornay"":0.434,""Christianity"":0.75,""James Corden"":0.069,""Hollywood"":1,""Stunt"":0.45},""_expires"":""2017-06-24T22:43:00.0008415Z""},{""XSentiment"":0.178008512,""_expires"":""2017-06-24T22:43:01.2750814Z""}]},{""_tag"":""cmplx$http://www.complex.com/music/2017/06/goldlink-releases-crew-remix-featuring-gucci-mane"",""i"":{""constant"":1,""id"":""cmplx$http://www.complex.com/music/2017/06/goldlink-releases-crew-remix-featuring-gucci-mane""},""j"":[{""_title"":""Gucci Mane Jumps on Remix of GoldLink's \""Crew\""""},{""RVisionTags"":{""text"":0.9904163},""SVisionAdult"":{""isAdultContent"":false,""isRacyContent"":false,""adultScore"":0.140676036,""racyScore"":0.08041213},""_expires"":""2017-06-25T03:29:03.5393778Z""},{""Emotion0"":{""anger"":0.09716808,""contempt"":0.00540762255,""disgust"":0.0220363177,""fear"":0.0168951228,""happiness"":0.00362249953,""neutral"":0.772761941,""sadness"":0.028037779,""surprise"":0.05407063},""Emotion1"":{""anger"":0.419406265,""contempt"":0.032055337,""disgust"":0.03494472,""fear"":0.004095346,""happiness"":0.218424827,""neutral"":0.2713901,""sadness"":0.0143981427,""surprise"":0.005285231},""_expires"":""2017-06-25T03:29:01.7568819Z""},{""Tags"":{""Gucci Mane"":0.993,""Goldlink"":0.246,""Joshua"":0.003,""Twitter Inc."":0.987,""Public Relations"":0.054,""Washington, D.C."":1,""Shy Glizzy"":0.149,""Turntablism"":0.004,""Shazam"":0.915,""Apple Music"":0.473,""You Can"":0.003,""iTunes"":0.956,""United States"":1,""Country"":0.996,""Miami Heat"":0.888,""Houston Rockets"":0.81,""Los Angeles"":1,""Portland Trail Blazers"":0.972,""Brooklyn Nets"":0.74,""Philadelphia"":1,""Monument Records"":0.173,""Go-go"":0.558,""Down"":0.126,""Album"":0.984,""Xavier Musketeers men's basketball"":0.011,""TEAM*"":0.006,""No Way Out"":0.007},""_expires"":""2017-06-25T03:29:01.4924568Z""},{""XSentiment"":0.7643385,""_expires"":""2017-06-25T03:29:02.1167452Z""}]},{""_tag"":""cmplx$http://www.complex.com/music/2017/06/snoop-dogg-mock-young-thug-and-lil-uzi-vert-moment-i-feared-video"",""i"":{""constant"":1,""id"":""cmplx$http://www.complex.com/music/2017/06/snoop-dogg-mock-young-thug-and-lil-uzi-vert-moment-i-feared-video""},""j"":[{""_title"":""It Looks Like Snoop Dogg Is Mocking Young Thug and Lil Uzi Vert in New Video for \""Moment I Feared\""""},{""RVisionTags"":{""person"":0.998487234},""SVisionAdult"":{""isAdultContent"":false,""isRacyContent"":false,""adultScore"":0.0242747813,""racyScore"":0.02104937},""_expires"":""2017-06-24T20:58:27.8900749Z""},{""Emotion0"":{""anger"":0.000228447025,""contempt"":0.0007477614,""disgust"":0.000145930273,""fear"":1.614125E-07,""happiness"":0.00138306723,""neutral"":0.997234,""sadness"":0.000220693677,""surprise"":3.99426281E-05},""_expires"":""2017-06-24T20:58:27.5944467Z""},{""Tags"":{""Snoop Dogg"":1,""Young Thug"":0.959,""Uzi"":0.67,""Vert (music producer)"":0.003,""New Video"":0.003,""Kyle Broflovski"":0.032,""Philadelphia"":0.006,""Twitter Inc."":1,""Subscription business model"":0.008,""Complex"":0.995,""WorldStarHipHop"":0.237,""Rick Rock"":0.449,""Fonzie"":0.026,""Hyphy"":0.228,""YouTube"":0.999,""300 Entertainment"":0.009,""I Do"":0.006,""Billboard Music Award for Woman of the Year"":0.007,""You Know What It Is"":0.007,""Beautiful"":1,""Hip hop music"":1},""_expires"":""2017-06-24T20:58:27.1518034Z""},{""XSentiment"":0.0003323581,""_expires"":""2017-06-24T20:58:28.2026657Z""}]},{""_tag"":""cmplx$http://www.complex.com/sports/2017/06/will-lebron-james-ever-be-the-goat-square-up"",""i"":{""constant"":1,""id"":""cmplx$http://www.complex.com/sports/2017/06/will-lebron-james-ever-be-the-goat-square-up""},""j"":[{""_title"":""Square Up: Will LeBron James Ever Be the G.O.A.T.?""},{""RVisionTags"":{""person"":0.996526659,""outdoor"":0.8773945,""player"":0.8584581,""athletic game"":0.8081653,""sport"":0.7153003,""green"":0.640901864},""SVisionAdult"":{""isAdultContent"":false,""isRacyContent"":false,""adultScore"":0.0165802147,""racyScore"":0.0130834375},""_expires"":""2017-06-24T17:04:57.3297536Z""},{""Emotion0"":{""anger"":0.03943785,""contempt"":0.05141066,""disgust"":0.0008398856,""fear"":3.766839E-05,""happiness"":0.00017841009,""neutral"":0.9013937,""sadness"":0.00586816063,""surprise"":0.0008336294},""_expires"":""2017-06-24T17:04:55.650471Z""},{""Tags"":{""Square, Inc."":0.005,""LeBron James"":1,""Complex"":0.926,""Twitter Inc."":0.991,""UNK NBA"":1,""Lil B"":0.415,""Cleveland Cavaliers"":1,""Golden State Warriors"":1},""_expires"":""2017-06-24T17:04:54.617989Z""},{""XSentiment"":0.00127977342,""_expires"":""2017-06-24T17:04:55.400445Z""}]},{""_tag"":""cmplx$http://www.complex.com/life/2017/06/cop-who-killed-philando-castile-says-smell-of-weed-made-him-fear-for-life"",""i"":{""constant"":1,""id"":""cmplx$http://www.complex.com/life/2017/06/cop-who-killed-philando-castile-says-smell-of-weed-made-him-fear-for-life""},""j"":[{""_title"":""Cop Who Killed Philando Castile Says Smell of Weed Made Him Fear for His Life""},{""RVisionTags"":{""sky"":0.998623848,""outdoor"":0.980991364},""SVisionAdult"":{""isAdultContent"":false,""isRacyContent"":false,""adultScore"":0.01796771,""racyScore"":0.0286844},""_expires"":""2017-06-24T16:33:36.9062355Z""},{""_expires"":""2017-06-24T16:33:36.2030935Z""},{""Tags"":{""Scared"":0.011,""Philadelphia"":0.693,""Twitter Inc."":1,""Ramsey County"":0.995,""County attorney"":0.084,""Minnesota"":1,""The Life of the Party"":0.009,""And I"":0.003,""Police"":0.238,""Dashcam"":0.092,""Murder"":0.673,""Complex"":0.508},""_expires"":""2017-06-24T16:33:35.8124592Z""},{""XSentiment"":4.440892E-16,""_expires"":""2017-06-24T16:33:36.4218429Z""}]},{""_tag"":""cmplx$http://www.complex.com/music/2017/06/a-history-of-bow-wow-fails"",""i"":{""constant"":1,""id"":""cmplx$http://www.complex.com/music/2017/06/a-history-of-bow-wow-fails""},""j"":[{""_title"":""A History of Bow Wow Taking L's""},{""RVisionTags"":{""person"":0.999946833,""crowd"":0.236136854},""SVisionAdult"":{""isAdultContent"":false,""isRacyContent"":false,""adultScore"":0.02204849,""racyScore"":0.0223767031},""_expires"":""2017-06-24T15:56:22.5001293Z""},{""Emotion0"":{""anger"":1.22613847E-05,""contempt"":0.0200665444,""disgust"":0.000140431745,""fear"":8.001536E-09,""happiness"":0.883786,""neutral"":0.0959461853,""sadness"":1.50497419E-06,""surprise"":4.703166E-05},""_expires"":""2017-06-24T15:56:21.7295682Z""},{""Tags"":{""Bow Wow"":1,""He Is"":0.003,""Forbes"":0.995,""Hip hop music"":1,""iTunes"":0.974,""Atlantic Ocean"":0.051,""Vibe"":0.982,""The Source"":0.991,""GQ"":1,""Esquire"":0.998,""Stephen Sondheim"":0.606,""Twitter Inc."":1,""Snoop Dogg"":1,""The Arsenio Hall Show"":0.043,""Kurtis Blow"":0.325,""Michael Jordan"":0.785,""Vine"":0.585,""Oh, hell"":0.005,""Rent"":0.155,""Los Angeles"":1,""Grammy Awards"":1,""Scuderia Ferrari"":0.099,""Instagram"":0.996,""Migos"":0.01,""December 8"":0.003,""Live television"":0.401,""Grammy Awards Ceremony"":0.102,""Timothy Sykes"":0.173,""President of the United States"":0.998,""Donald Trump"":0.069,""Pacific Time Zone"":0.278,""Funkmaster Flex"":0.375,""Jermaine Dupri"":0.996,""Complex"":0.959},""_expires"":""2017-06-24T15:56:22.1957887Z""},{""XSentiment"":1,""_expires"":""2017-06-24T15:56:22.336416Z""}]},{""_tag"":""cmplx$http://www.complex.com/style/2017/06/how-rhude-one-of-best-los-angeles-brands-started-with-a-single-t-shirt"",""i"":{""constant"":1,""id"":""cmplx$http://www.complex.com/style/2017/06/how-rhude-one-of-best-los-angeles-brands-started-with-a-single-t-shirt""},""j"":[{""_title"":""How Rhude, One of the Best L.A. Brands, Started With a Single T-Shirt ""},{""RVisionTags"":{""person"":0.993622959,""floor"":0.989562333,""indoor"":0.98370254,""standing"":0.810940444,""curtain"":0.7955723,""suit"":0.335544765},""SVisionAdult"":{""isAdultContent"":false,""isRacyContent"":false,""adultScore"":0.08148283,""racyScore"":0.0423882678},""_expires"":""2017-06-24T12:04:37.9596756Z""},{""Emotion0"":{""anger"":0.0006152864,""contempt"":0.00599214761,""disgust"":0.0002703283,""fear"":1.61605785E-05,""happiness"":0.00174113235,""neutral"":0.984110057,""sadness"":0.006996317,""surprise"":0.0002585811},""Emotion1"":{""anger"":3.47017163E-07,""contempt"":0.00013010086,""disgust"":2.59245485E-06,""fear"":1.34914092E-06,""happiness"":0.000740572,""neutral"":0.997228861,""sadness"":0.00182852661,""surprise"":6.765087E-05},""_expires"":""2017-06-24T12:04:37.3269255Z""},{""Tags"":{""The Best"":0.013,""Starting pitcher"":0.489,""T-shirt"":0.041,""Twitter Inc."":0.955,""Mixmaster Spade"":0.977,""Kendrick Lamar"":1,""Snoop Dogg"":1,""BET Awards"":0.014,""ASAP Rocky"":0.979,""Kevin Durant"":0.023,""Jimmy Butler"":0.052,""Barneys New York"":0.011,""Patron saint"":0.034,""United States"":1,""Big Sean"":0.986,""Manila"":0.676,""Culture of the Philippines"":0.938,""Trade in Services Agreement"":0.006,""Kanye West"":1,""Arnold Schwarzenegger"":0.004,""Comme des Garçons"":0.987,""Rei Kawakubo"":0.911,""Sugar Land"":0.671,""Texas"":1,""Posttraumatic stress disorder"":0.02,""Earth, Wind & Fire"":0.003,""Mike Jones"":0.007,""White American"":0.012,""Prada Marfa"":1,""Elmgreen and Dragset"":0.992,""Complex"":0.992},""_expires"":""2017-06-24T12:04:36.8672077Z""},{""XSentiment"":1,""_expires"":""2017-06-24T12:04:37.5488193Z""}]},{""_tag"":""cmplx$http://www.complex.com/music/2017/06/bow-wow-gets-backlash-over-sexist-instagram-caption"",""i"":{""constant"":1,""id"":""cmplx$http://www.complex.com/music/2017/06/bow-wow-gets-backlash-over-sexist-instagram-caption""},""j"":[{""_title"":""The Internet Blasts Bow Wow for His Message About Women's Behavior""},{""RVisionTags"":{""person"":0.962055564,""man"":0.9547968},""SVisionAdult"":{""isAdultContent"":false,""isRacyContent"":false,""adultScore"":0.0112607479,""racyScore"":0.0106668333},""TVisionCelebrities"":{""Bow Wow"":0.9601661},""_expires"":""2017-06-24T01:43:32.5645689Z""},{""Emotion0"":{""anger"":0.0011651055,""contempt"":0.005131879,""disgust"":0.000264819653,""fear"":2.01475978E-05,""happiness"":0.000101240934,""neutral"":0.98257935,""sadness"":0.0105332062,""surprise"":0.000204226963},""_expires"":""2017-06-24T01:43:31.8143804Z""},{""Tags"":{""The Internet"":0.661,""Bow Wow"":0.398,""Twitter Inc."":1,""Instagram"":0.998,""I Wonder Why"":0.004,""Sexism"":0.362},""_expires"":""2017-06-24T01:43:31.5472109Z""},{""XSentiment"":0.99999994,""_expires"":""2017-06-24T01:43:32.2439044Z""}]}]},""p"":[0.8153846,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154,0.0153846154],""VWState"":{""m"":""4c5bcf8eb1ef4d3ba327dacba2f336a1/4c5bcf8eb1ef4d3ba327dacba2f336a1""}}"; using (var vw = new VowpalWabbit("--cb_adf")) { var obj = JsonConvert.DeserializeObject(json); var bytes = new byte[Encoding.UTF8.GetMaxByteCount(json.Length) + 1]; var byteLen = Encoding.UTF8.GetBytes(json, 0, json.Length, bytes, 0) + 1;// trailing \0 VowpalWabbitDecisionServiceInteractionHeader header; List examples = null; try { examples = vw.ParseDecisionServiceJson(bytes, 0, byteLen, copyJson: false, header: out header); Assert.AreEqual("7cacacea2c6e49b5b922f6f517a325ed", header.EventId); CollectionAssert.AreEqual(new[] { 9, 4, 13, 10, 8, 5, 2, 3, 12, 11, 7, 6, 1 }, header.Actions, "Actions mismatch"); CollectionAssert.AreEqual(new[] { 0.8153846f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f, 0.0153846154f }, header.Probabilities, "Probabilities mismatch"); Assert.AreEqual(0, header.ProbabilityOfDrop); Assert.AreEqual(14, examples.Count); } finally { if (examples != null) foreach (var ex in examples) if (ex != null) ex.Dispose(); } } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestPartialJson() { using (var validator = new VowpalWabbitExampleJsonValidator(new VowpalWabbitSettings { Arguments = "--cb_adf", PropertyConfiguration = new PropertyConfiguration { MultiProperty = "adf", TextProperty = "someText", FeatureIgnorePrefix = "xxx" } })) { var json = "{\"eventId\":123,\"c\":{\"Age\":25,\"adf\":[{\"someText\":\"w1 w2\", \"a\":{\"x\":1}, \"xxxxIgnoreMe\":2}, {\"someText\":\"w2 w3\"}], \"_labelIndex\":1, \"_label_Cost\":-1, \"_label_Probability\":0.3},\"post\":456}"; using (var textReader = new JsonTextReader(new StringReader(json))) { textReader.Read(); Assert.AreEqual(JsonToken.StartObject, textReader.TokenType); textReader.Read(); Assert.AreEqual(JsonToken.PropertyName, textReader.TokenType); Assert.AreEqual("eventId", textReader.Value); textReader.Read(); Assert.AreEqual(JsonToken.Integer, textReader.TokenType); Assert.AreEqual((Int64)123, textReader.Value); textReader.Read(); Assert.AreEqual(JsonToken.PropertyName, textReader.TokenType); Assert.AreEqual("c", textReader.Value); textReader.Read(); validator.Validate(new[] { "shared | Age:25", " | w1 w2 |a x:1", "0:-1:.3 | w2 w3" }, textReader, VowpalWabbitLabelComparator.ContextualBandit); textReader.Read(); Assert.AreEqual(JsonToken.PropertyName, textReader.TokenType); Assert.AreEqual("post", textReader.Value); textReader.Read(); Assert.AreEqual((Int64)456, textReader.Value); } } } public class MyContext { [Feature] public int Feature { get; set; } [JsonProperty("_multi")] public IEnumerable Multi { get; set; } } public class MyADF { [Feature] public int Foo { get; set; } } [TestMethod] public void TestNumADFs() { var jsonDirectSerializer = VowpalWabbitSerializerFactory.CreateSerializer(new VowpalWabbitSettings { TypeInspector = JsonTypeInspector.Default }) as IVowpalWabbitMultiExampleSerializerCompiler; Assert.IsNotNull(jsonDirectSerializer); Assert.AreEqual(3, jsonDirectSerializer.GetNumberOfActionDependentExamples(new MyContext { Multi = new MyADF[3] })); } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestJsonDict.cs000066400000000000000000000233261323713302100224760ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using Newtonsoft.Json; using Newtonsoft.Json.Serialization; using System; using System.Collections.Concurrent; using System.Collections.Generic; using System.Globalization; using System.Linq; using System.Text; using System.Threading; using System.Threading.Tasks; using VW; using VW.Serializer; namespace cs_unittest { [TestClass] public class TestJsonDictClass { public class Context { public Context(float[] vector, int id, JsonSerializerSettings settings) { this.Vector = vector; this.Id = id; this.JSON = JsonConvert.SerializeObject(this, settings); var sb = new StringBuilder(); sb.AppendFormat("| Id:{0}", this.Id); foreach (var v in vector) sb.AppendFormat(CultureInfo.InvariantCulture, " :{0}", v); this.VW = sb.ToString(); } [JsonProperty(IsReference = true)] public float[] Vector { get; private set; } [JsonProperty] public int Id { get; private set; } [JsonIgnore] public string VW { get; set; } [JsonIgnore] public string JSON { get; set; } } [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestJsonDict() { var vec = new float[] { 1, 2, 3 }; var jsonResolver = new RefResolve(); var settings = new JsonSerializerSettings { ReferenceResolverProvider = () => jsonResolver }; var ctx1 = new Context(vec, 1, settings); var ctx2 = new Context(vec, 2, settings); using (var vw = new VowpalWabbit(new VowpalWabbitSettings { EnableStringExampleGeneration = true })) using (var resolver = new VowpalWabbitJsonReferenceResolver(serializer => Assert.Fail())) using (var serializer1 = new VowpalWabbitJsonSerializer(vw, resolver)) using (var example1 = serializer1.ParseAndCreate(ctx1.JSON)) using (var serializer2 = new VowpalWabbitJsonSerializer(vw, resolver)) using (var example2 = serializer2.ParseAndCreate(ctx2.JSON)) using (var validator = new VowpalWabbitExampleJsonValidator()) { validator.Validate("| Id:1 :1 :2 :3", example1); validator.Validate(ctx1.VW, example1); validator.Validate("| Id:2 :1 :2 :3", example2); validator.Validate(ctx2.VW, example2); } } [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestJsonDictReverse() { var vec = new float[] { 1, 2, 3 }; var jsonResolver = new RefResolve(); var settings = new JsonSerializerSettings { ReferenceResolverProvider = () => jsonResolver }; var ctx1 = new Context(vec, 1, settings); var ctx2 = new Context(vec, 2, settings); VowpalWabbitJsonSerializer delayedSerializer = null; using (var validator = new VowpalWabbitExampleJsonValidator()) using (var vw = new VowpalWabbit(new VowpalWabbitSettings { EnableStringExampleGeneration = true })) using (var resolver = new VowpalWabbitJsonReferenceResolver(serializer => delayedSerializer = serializer)) { var serializer2 = new VowpalWabbitJsonSerializer(vw, resolver); var example2 = serializer2.ParseAndCreate(ctx2.JSON); // incomplete data Assert.IsNull(example2); // triggers example2 completion using (var serializer1 = new VowpalWabbitJsonSerializer(vw, resolver)) using (var example1 = serializer1.ParseAndCreate(ctx1.JSON)) { validator.Validate("| Id:1 :1 :2 :3", example1); } Assert.IsNotNull(delayedSerializer); using (var delayedExample2 = delayedSerializer.CreateExamples()) { validator.Validate("| Id:2 :1 :2 :3", delayedExample2); } delayedSerializer.Dispose(); } } [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestJsonDictThreading() { var jsonResolver = new RefResolve(); var settings = new JsonSerializerSettings { ReferenceResolverProvider = () => jsonResolver }; var rnd = new Random(123); var examples = new List(); var id = 0; // different reference objects for (int i = 0; i < 10; i++) { var data = Enumerable.Range(1, 5).Select(_ => (float)rnd.Next(10)).ToArray(); // referencing the same data for (int j = 0; j < 5; j++) examples.Add(new Context(data, id++, settings)); } for (int i = 0; i < 4; i++) { Permute(examples, rnd); for (int maxDegreeOfParallelism = 1; maxDegreeOfParallelism < 4; maxDegreeOfParallelism++) { var examplesFound = 0; using (var vw = new VowpalWabbit(new VowpalWabbitSettings { EnableStringExampleGeneration = true, EnableThreadSafeExamplePooling = true })) using (var resolver = new VowpalWabbitJsonReferenceResolver(serializer => { using (var example = serializer.CreateExamples()) { ValidateExample(example, (Context)serializer.UserContext); } serializer.Dispose(); Interlocked.Increment(ref examplesFound); })) { Parallel.ForEach( Partitioner.Create(0, examples.Count), new ParallelOptions { MaxDegreeOfParallelism = maxDegreeOfParallelism }, range => { for (int j = range.Item1; j < range.Item2; j++) { var ctx = examples[j]; var serializer = new VowpalWabbitJsonSerializer(vw, resolver) { UserContext = ctx }; var example = serializer.ParseAndCreate(ctx.JSON); // example not ready yet if (example == null) continue; ValidateExample(example, ctx); example.Dispose(); serializer.Dispose(); Interlocked.Increment(ref examplesFound); } }); } Assert.AreEqual(examples.Count, examplesFound); } } } public void ValidateExample(VowpalWabbitExampleCollection example, Context ctx) { using (var vw = new VowpalWabbit(new VowpalWabbitSettings { EnableStringExampleGeneration = true })) using (var validator = new VowpalWabbitExampleJsonValidator()) { var singleExample = (VowpalWabbitSingleLineExampleCollection)example; validator.Validate(ctx.VW, example); } } public static void Permute(List arr, Random rnd) { for (int i = 0; i < arr.Count - 1; i++) { int swapIndex = rnd.Next(i, arr.Count); T temp = arr[swapIndex]; arr[swapIndex] = arr[i]; arr[i] = temp; } } public class RefResolve : IReferenceResolver { private readonly IDictionary data; private readonly IDictionary otherData; private class ReferenceEqualityComparer : IEqualityComparer { bool IEqualityComparer.Equals(object x, object y) { return object.ReferenceEquals(x, y); } public int GetHashCode(object obj) { return obj.GetHashCode(); } } public RefResolve() { this.data = new Dictionary(new ReferenceEqualityComparer()); this.otherData = new Dictionary(); } public object ResolveReference(object context, string reference) { return this.otherData[reference]; } public string GetReference(object context, object value) { foreach (var kv in this.otherData) { if (object.ReferenceEquals(kv.Value, value)) return kv.Key; } var id = Guid.NewGuid().ToString(); this.AddReference(null, id, value); return id; } public bool IsReferenced(object context, object value) { return this.otherData.Any(kv => object.ReferenceEquals(kv.Value, value)) || this.data.ContainsKey(value); } public void AddReference(object context, string reference, object value) { this.otherData.Add(reference, value); this.data.Add(value, value); } } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestJsonDirect.cs000066400000000000000000000371201323713302100230220ustar00rootroot00000000000000// -------------------------------------------------------------------------------------------------------------------- // // Copyright (c) by respective owners including Yahoo!, Microsoft, and // individual contributors. All rights reserved. Released under a BSD // license as described in the file LICENSE. // // -------------------------------------------------------------------------------------------------------------------- using Microsoft.VisualStudio.TestTools.UnitTesting; using Newtonsoft.Json; using System.Collections.Generic; using System.Linq; using VW; using VW.Labels; using VW.Serializer; namespace cs_unittest { [TestClass] public class TestJsonDirectClass { [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonDirect() { using (var vw = new VowpalWabbitExampleValidator(new VowpalWabbitSettings { TypeInspector = JsonTypeInspector.Default })) { vw.Validate("| Clicks:5 |a Bar:1 Age25_old |b Marker", new JsonContext() { Ns1 = new Namespace1 { Foo = 1, Age = "25 old", DontConsider = "XXX" }, Ns2 = new Namespace2 { FeatureA = true }, Clicks = 5 }); vw.Validate("| Clicks:5 |a Bar:1", new JsonContext() { Ns1 = new Namespace1 { Foo = 1, DontConsider = "XXX" }, Clicks = 5, IgnoreMe = "true" }); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonDirectWithLabel() { using (var vw = new VowpalWabbitExampleValidator(new VowpalWabbitSettings { TypeInspector = JsonTypeInspector.Default })) { vw.Validate("13 | Clicks:5 MoreClicks:3", new JsonContext() { Label = new SimpleLabel { Label = 13 }, Clicks = 5, MoreClicks = 3, IgnoreMe2 = "YYY" }); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonOptIn() { using (var vw = new VowpalWabbitExampleValidator(new VowpalWabbitSettings { TypeInspector = JsonTypeInspector.Default })) { vw.Validate("| Clicked |Ns2 Marker", new JsonContextOptIn() { Clicked = true, IgnoredNamespace = new Namespace1 { Foo = 3 }, Ns2 = new Namespace2 { FeatureA = true } }); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestLabelJsonSerialization() { Assert.AreEqual( "{\"_label\":{\"Label\":25.0},\"Clicks\":0,\"MoreClicks\":3}", JsonConvert.SerializeObject(new JsonContext() { Label = new SimpleLabel { Label = 25 }, MoreClicks = 3, IgnoreMe = "XXX" })); } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonArray() { using (var vw = new VowpalWabbitExampleValidator(new VowpalWabbitSettings { TypeInspector = JsonTypeInspector.Default })) { vw.Validate("1:2:.5 |Data :.1 :.2 :.3", new JsonContextArray() { Label = new ContextualBanditLabel { Action = 1, Cost = 2, Probability = .5f }, Data = new[] { .1f, .2f, .3f } }); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] [TestCategory("Vowpal Wabbit/Marshal")] public void TestJsonDictionaryStringFloat() { using (var vw = new VowpalWabbitExampleValidator(new VowpalWabbitSettings { TypeInspector = JsonTypeInspector.Default })) { vw.Validate("|Features Feature1:2.1 Feature2:3.2", new POCODict { Features = new Dictionary { { "Feature1", 2.1f }, { "Feature2", 3.2f } } }); } // test serialzier caching too using (var vw = new VowpalWabbitExampleValidator(new VowpalWabbitSettings { TypeInspector = TypeInspector.All })) { vw.Validate("| Abc:2.1 def:3.2", new POCODict { Features = new Dictionary { { "Abc", 2.1f }, { "def", 3.2f } } }); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonByte() { using (var vw = new VowpalWabbitExampleValidator(new VowpalWabbitSettings { TypeInspector = JsonTypeInspector.Default })) { vw.Validate("| Feature:25", new JsonContextByte { Feature = 25 }); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonDirectText() { using (var vw = new VowpalWabbitExampleValidator(new VowpalWabbitSettings { TypeInspector = JsonTypeInspector.Default })) { vw.Validate("| a b c |a d e f", new JsonText { Text = "a b c", AuxInfo = "Foo", A = new JsonText { Text = "d e f", AuxInfo = "Bar" } }); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonDirectMulti() { using (var vw = new VowpalWabbitExampleValidator(new VowpalWabbitSettings("--cb_adf") { TypeInspector = JsonTypeInspector.Default })) { vw.Validate(new[] { "shared | Ageteen", " | Id:1", " | Id:2" }, new JsonShared { Age = "teen", Documents = new[] { new JsonADF { Id = 1 }, new JsonADF { Id = 2 } } }); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonDirectMultiList() { using (var vw = new VowpalWabbitExampleValidator(new VowpalWabbitSettings("--cb_adf") { TypeInspector = JsonTypeInspector.Default })) { vw.Validate(new[] { " | Id:1", " | Id:2" }, new JsonSharedList { _multi = new List { new JsonADF { Id = 1 }, new JsonADF { Id = 2 } } }); vw.Validate(new[] { "shared | Ageteen", " | Id:1" }, new JsonSharedList { Age = "teen", _multi = new List { new JsonADF { Id = 1 } } }); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonDirectMultiEmpty() { using (var vw = new VowpalWabbitExampleValidator(new VowpalWabbitSettings { TypeInspector = JsonTypeInspector.Default })) { vw.Validate(new[] { " | Id:1", " | Id:2" }, new JsonSharedEmpty { Age = "ignored", _multi = new[] { new JsonADF { Id = 1 }, new JsonADF { Id = 2 } } }); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonConvertibleMulti() { using (var vw = new VowpalWabbitExampleValidator(new VowpalWabbitSettings("--cb_adf") { TypeInspector = JsonTypeInspector.Default })) { vw.Validate(new[] { "shared | Bar:5", " | Foo:1 |Value test:1.2", " | Foo:2 |Value test:2.3", " | Foo:3 |Value titleabc\"def", }, new JsonRawAdfString { Bar = 5, _multi = new[] { new JsonRawString { Foo = 1, Value = JsonConvert.SerializeObject(new { test = 1.2 }) }, new JsonRawString { Foo = 2, Value = JsonConvert.SerializeObject(new { test = 2.3 }) }, new JsonRawString { Foo = 3, Value = JsonConvert.SerializeObject(new { title = "abc\"def", _ignoreMe = 1 }) }, } }); var adf = new JsonRawString { Foo = 1, Value = JsonConvert.SerializeObject(new { A = new { test = 1.2 }, B = new { bar = 2 } }), Values = new[] { JsonConvert.SerializeObject(new { D = new { d = 1.2 } }), JsonConvert.SerializeObject(new { E = new { e = true } }), }.ToList() }; var ctx = new JsonRawAdfString { Bar = 5, _multi = new[] { adf } }; vw.Validate(new[] { "shared | Bar:5", " | Foo:1 |A test:1.2 |B bar:2 |D d:1.2 |E e" }, ctx); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonConvertible() { using (var vw = new VowpalWabbitExampleValidator(new VowpalWabbitSettings("") { TypeInspector = JsonTypeInspector.Default })) { var adf = new JsonRawString { Foo = 1, Value = JsonConvert.SerializeObject(new { A = new { test = 1.2 }, B = new { bar = 2 } }), Values = new[] { JsonConvert.SerializeObject(new { D = new { d = 1.2 } }), JsonConvert.SerializeObject(new { E = new { e = true } }), JsonConvert.SerializeObject(new { F = new { title = "abc\"def" } }), }.ToList() }; vw.Validate(" | Foo:1 |A test:1.2 |B bar:2 |D d:1.2 |E e |F titleabc\"def", adf); } } [TestMethod] [TestCategory("Vowpal Wabbit/JSON")] public void TestJsonDictArray() { using (var vw = new VowpalWabbitExampleValidator(new VowpalWabbitSettings(string.Empty) { TypeInspector = JsonTypeInspector.Default })) { var ex = new JsonDictArray { Features = new Dictionary { { "A", new float[] { 1, 2, 3.1f} }, { "B", new float[] { 2, 3, 4.1f} } } }; vw.Validate(" |A :1 :2 :3.1 |B :2 :3 :4.1", ex); } } } public class JsonRawString { public int Foo { get; set; } [JsonConverter(typeof(JsonRawStringConverter))] public string Value { get; set; } [JsonConverter(typeof(JsonRawStringListConverter))] public List Values { get; set; } } public class JsonRawAdfString { public int Bar { get; set; } public JsonRawString[] _multi { get; set; } } public class JsonText { [JsonProperty("_text")] public string Text { get; set; } [JsonProperty("_auxInfo")] public string AuxInfo { get; set; } [JsonProperty("a")] public JsonText A { get; set; } } [JsonObject(MemberSerialization = MemberSerialization.OptOut)] public class JsonContextArray { [JsonIgnore] public ILabel Label { get; set; } public float[] Data { get; set; } } [JsonObject(MemberSerialization = MemberSerialization.OptIn)] public class JsonContextOptIn { public int IgnoreMe { get; set; } [JsonProperty] public bool Clicked { get; set; } public Namespace1 IgnoredNamespace { get; set; } [JsonProperty] public Namespace2 Ns2 { get; set; } } public class JsonContext { [JsonProperty(PropertyName = "_label")] public SimpleLabel Label { get; set; } [JsonProperty(PropertyName = "a", NullValueHandling = NullValueHandling.Ignore)] public Namespace1 Ns1 { get; set; } [JsonProperty(PropertyName = "b", NullValueHandling = NullValueHandling.Ignore)] public Namespace2 Ns2 { get; set; } [JsonProperty] public int Clicks { get; set; } public int MoreClicks { get; set; } [JsonIgnore] public object IgnoreMe { get; set; } [JsonProperty(PropertyName = "_aux", NullValueHandling = NullValueHandling.Ignore)] public object IgnoreMe2 { get; set; } } public class JsonContextByte { public byte Feature { get; set; } } public class Namespace1 { [JsonProperty(PropertyName = "Bar", NullValueHandling = NullValueHandling.Ignore)] public int Foo { get; set; } [JsonProperty] public string Age { get; set; } [JsonIgnore] public string DontConsider { get; set; } [JsonProperty] public string EscapeCharacterString { get; set; } [JsonProperty("_text")] public string EscapeCharactersText { get; set; } } public class Namespace2 { [JsonProperty("Marker")] public bool FeatureA { get; set; } } public class JsonShared { public string Age { get; set; } [JsonProperty("_multi")] public JsonADF[] Documents { get; set; } } public class JsonSharedList { public string Age { get; set; } public List _multi { get; set; } } public class JsonSharedEmpty { [JsonProperty("_ignoreMe")] public string Age { get; set; } public IEnumerable _multi { get; set; } } public class JsonADF { public int Id { get; set; } } public class JsonDictArray { public Dictionary Features { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestLabels.cs000066400000000000000000000056311323713302100221620ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using VW; using VW.Labels; using VW.Serializer; using VW.Serializer.Attributes; namespace cs_unittest { [TestClass] public class TestLabelsClass { [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] [TestCategory("Vowpal Wabbit")] public void TestLabels() { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("3.2 | Feature:25", new SimpleContext { Feature = 25, Label = new SimpleLabel { Label = 3.2f } }); vw.Validate("| Feature:25", new SimpleContext { Feature = 25, }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] [TestCategory("Vowpal Wabbit")] public void TestLabelsNoAnnotation() { using (var vw = new VowpalWabbitExampleValidator( new VowpalWabbitSettings { TypeInspector = TypeInspector.All })) { vw.Validate("3.2 | Feature:25", new SimpleContextNoAnnotation { Feature = 25, Label = new SimpleLabel { Label = 3.2f } }); vw.Validate("| Feature:25", new SimpleContextNoAnnotation { Feature = 25, }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] [TestCategory("Vowpal Wabbit")] public void TestStringLabels() { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("3.2 | Feature:25", new SimpleStringContext { Feature = 25, Label = "3.2" }); vw.Validate("| Feature:25", new SimpleStringContext { Feature = 25, }); } } } public class SimpleContext { [Feature] public int Feature { get; set; } [Label] public ILabel Label { get; set; } } public class SimpleContextNoAnnotation { public int Feature { get; set; } public ILabel Label { get; set; } } public class SimpleStringContext { [Feature] public int Feature { get; set; } [Label] public string Label { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestLeak.cs000066400000000000000000000004521323713302100216300ustar00rootroot00000000000000using VW; namespace cs_unittest { public class TestLeakClass { public void Leak() { #if DEBUG VowpalWabbitLeakTest.Leak(); #endif } public void NoLeak() { #if DEBUG VowpalWabbitLeakTest.NoLeak(); #endif } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestManagedHash.cs000066400000000000000000000125551323713302100231230ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Diagnostics; using System.Text; using TrainSet0002Dat; using VW; namespace cs_unittest { [TestClass] public class TestManagedHash { static string[] data = { "wLceNLHVtpuZYtPwPaQ", "949007 ", "Kvq7Hj6RSImhZUhcZuL", "KDqmcZO2h6CIM1j ", "656656 ", "X8bRcLpb8yrIkA2A ", "k5USpack9N ", "Okv90y9lOamog3qXRIk", "860628 ", "903251 ", "289149 ", "727293 ", "660473 ", "ciajGNL930GlKi5b ", "744202 ", "458807 ", "466125 ", "101824 ", "0RVP8HWyKGFjkJG8RA ", "mm3XQ0ZKJQ4rPmtNN ", "ZL7Z6bzFVsL0VQGe5ss", "690592 ", "Apbr4WNUDHmL7OWxm ", "342052 ", "286245 ", "JbIIXVVbS3Y79uj4iI ", "S9E90IvFAVt ", "z2QWOpzi63 ", "gnCClcujq79 ", "hOZPaw9s4922I3S ", "mFWZjVtCOiymM2 ", "m6a93w7IRLNaadJbL ", "758870 ", "164290 ", "971935 ", "MVObGSH9iWxiyvp ", "135400 ", "T2b9WalhX9c ", "CQaS6KtGArRLtM5v ", "B0lNkkeP57ZLJjZAwfP", "695049 ", "BSbUX2YPm1daHvo6 ", "ReVgoh7mtQpghPDl ", "I0RnHRdk5IRFHJZaZST", "489901 ", "a9IZGkY6WLtX0X37D ", "061731 ", "402102 ", "IgRGpl2Z0OdgNzr6AH ", "tNlzNvlPQ0hXFlzjpe8", "m2JmhQ8L6DEnauuvSst", "141010 ", "534087 ", "599686 ", "000093 ", "707313 ", "563622 ", "HlcM6fNDjW ", "4qEn6lfmhd2b6Fo ", "ph5x9nJTFV ", "783062 ", "403127 ", "fGbvUKatET3SAf0rfA ", "IduDv41Z1z7Opirz ", "625285 ", "HbsPUqTZvWHI4ylB ", "554240 ", "849636 ", "1ElP3So1fCS ", "539836 ", "jELB4FrYkqwpmecr ", "Ko4EWBb3gFqN0PR7pvf", "VMX4dVyfAZ0V9VwK ", "K0BYm86Zg8PogMNSo ", "ajfcoff0sqt ", "373791 ", "220160 ", "dxZoyNeZZMiO ", "286375 ", "DEy4nNiHHd9nN ", "3gverMSb6ANY3wLj ", "ATGPA40OShUer ", "548754 ", "7NdgIl223apO ", "aaas hu as 撒 asfasd 阿萨", "oof Ồ hử hị ộ ở ỗ õ ẽ uyễn \r \t \n \\ ", "934625 ", "123" }; [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestHash() { InternalTestHash(""); InternalTestHash("--hash all"); InternalTestHash("--hash strings"); } [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestHashSpace() { using (var vw = new VowpalWabbit("")) { Assert.AreEqual(0ul, vw.HashSpace(" ")); Assert.AreEqual(0ul, vw.HashSpace("0")); } } [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestHashUnicodeSpace() { using (var vw = new VowpalWabbit("")) { var value = "ArticleTitleThe_25_Biggest_Art_Moments_of_2012" + (char)160; // Encoding.UTF8.GetMaxByteCount var nativeHash = vw.HashSpaceNative(value); var managedHash = vw.HashSpace(value); Assert.AreEqual(nativeHash, managedHash); } } private void InternalTestHash(string args) { var stopWatchNative = new Stopwatch(); var stopWatchManaged = new Stopwatch(); using (var vw = new VowpalWabbit(args)) { for (int i = 0; i < 10000; i++) { foreach (var item in data) { stopWatchNative.Start(); var nativeHash = vw.HashSpaceNative(item); stopWatchNative.Stop(); stopWatchManaged.Start(); var managedHash = vw.HashSpace(item); stopWatchManaged.Stop(); Assert.AreEqual(nativeHash, managedHash, item); } } } Console.WriteLine("Args: " + args); Console.WriteLine("native: {0}", stopWatchNative.Elapsed); Console.WriteLine("managed: {0}", stopWatchManaged.Elapsed); } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestMarshalNumeric.cs000066400000000000000000001616311323713302100236750ustar00rootroot00000000000000 using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using VW.Serializer; using VW.Serializer.Attributes; namespace cs_unittest { [TestClass] public class TestMarshalNumeric { [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericByte() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| Value:25", new NumericExampleByte() { Value = 25 }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericByteArray() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| :4 :2 :3", new NumericExampleByteArray() { Value = new System.Byte[] { 4, 2, 3 } }); vw.Validate("", new NumericExampleByteArray()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericByteArrayAnchor() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| :1 :4 :2 :3", new NumericExampleByteArrayAnchor() { Value = new System.Byte[] { 4, 2, 3 } }); vw.Validate("", new NumericExampleByteArrayAnchor()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericSByte() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| Value:25", new NumericExampleSByte() { Value = 25 }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericSByteArray() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| :4 :2 :3", new NumericExampleSByteArray() { Value = new System.SByte[] { 4, 2, 3 } }); vw.Validate("", new NumericExampleSByteArray()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericSByteArrayAnchor() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| :1 :4 :2 :3", new NumericExampleSByteArrayAnchor() { Value = new System.SByte[] { 4, 2, 3 } }); vw.Validate("", new NumericExampleSByteArrayAnchor()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericInt16() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| Value:25", new NumericExampleInt16() { Value = 25 }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericInt16Array() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| :4 :2 :3", new NumericExampleInt16Array() { Value = new System.Int16[] { 4, 2, 3 } }); vw.Validate("", new NumericExampleInt16Array()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericInt16ArrayAnchor() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| :1 :4 :2 :3", new NumericExampleInt16ArrayAnchor() { Value = new System.Int16[] { 4, 2, 3 } }); vw.Validate("", new NumericExampleInt16ArrayAnchor()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericInt32() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| Value:25", new NumericExampleInt32() { Value = 25 }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericInt32Array() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| :4 :2 :3", new NumericExampleInt32Array() { Value = new System.Int32[] { 4, 2, 3 } }); vw.Validate("", new NumericExampleInt32Array()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericInt32ArrayAnchor() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| :1 :4 :2 :3", new NumericExampleInt32ArrayAnchor() { Value = new System.Int32[] { 4, 2, 3 } }); vw.Validate("", new NumericExampleInt32ArrayAnchor()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericUInt16() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| Value:25", new NumericExampleUInt16() { Value = 25 }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericUInt16Array() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| :4 :2 :3", new NumericExampleUInt16Array() { Value = new System.UInt16[] { 4, 2, 3 } }); vw.Validate("", new NumericExampleUInt16Array()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericUInt16ArrayAnchor() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| :1 :4 :2 :3", new NumericExampleUInt16ArrayAnchor() { Value = new System.UInt16[] { 4, 2, 3 } }); vw.Validate("", new NumericExampleUInt16ArrayAnchor()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericUInt32() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| Value:25", new NumericExampleUInt32() { Value = 25 }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericUInt32Array() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| :4 :2 :3", new NumericExampleUInt32Array() { Value = new System.UInt32[] { 4, 2, 3 } }); vw.Validate("", new NumericExampleUInt32Array()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericUInt32ArrayAnchor() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| :1 :4 :2 :3", new NumericExampleUInt32ArrayAnchor() { Value = new System.UInt32[] { 4, 2, 3 } }); vw.Validate("", new NumericExampleUInt32ArrayAnchor()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericSingle() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| Value:25", new NumericExampleSingle() { Value = 25 }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericSingleArray() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| :4 :2 :3", new NumericExampleSingleArray() { Value = new System.Single[] { 4, 2, 3 } }); vw.Validate("", new NumericExampleSingleArray()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericSingleArrayAnchor() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| :1 :4 :2 :3", new NumericExampleSingleArrayAnchor() { Value = new System.Single[] { 4, 2, 3 } }); vw.Validate("", new NumericExampleSingleArrayAnchor()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericInt64() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| Value:25", new NumericExampleInt64() { Value = 25 }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericInt64Array() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| :4 :2 :3", new NumericExampleInt64Array() { Value = new System.Int64[] { 4, 2, 3 } }); vw.Validate("", new NumericExampleInt64Array()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericInt64ArrayAnchor() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| :1 :4 :2 :3", new NumericExampleInt64ArrayAnchor() { Value = new System.Int64[] { 4, 2, 3 } }); vw.Validate("", new NumericExampleInt64ArrayAnchor()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericUInt64() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| Value:25", new NumericExampleUInt64() { Value = 25 }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericUInt64Array() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| :4 :2 :3", new NumericExampleUInt64Array() { Value = new System.UInt64[] { 4, 2, 3 } }); vw.Validate("", new NumericExampleUInt64Array()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericUInt64ArrayAnchor() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| :1 :4 :2 :3", new NumericExampleUInt64ArrayAnchor() { Value = new System.UInt64[] { 4, 2, 3 } }); vw.Validate("", new NumericExampleUInt64ArrayAnchor()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericDouble() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| Value:25", new NumericExampleDouble() { Value = 25 }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericDoubleArray() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| :4 :2 :3", new NumericExampleDoubleArray() { Value = new System.Double[] { 4, 2, 3 } }); vw.Validate("", new NumericExampleDoubleArray()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumericDoubleArrayAnchor() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| :1 :4 :2 :3", new NumericExampleDoubleArrayAnchor() { Value = new System.Double[] { 4, 2, 3 } }); vw.Validate("", new NumericExampleDoubleArrayAnchor()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryByte() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleByte() { Dict = new Dictionary() }; example.Dict.Add(15, .5f); example.Dict.Add(5, .3f); example.Dict.Add(20, 123.2f); vw.Validate("| 15:0.5 5:0.3 20:123.2", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryByteString() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleByteString() { Dict = new Dictionary() }; example.Dict.Add(15, "0.5"); example.Dict.Add(5, "0.3"); example.Dict.Add(20, "123.2"); vw.Validate("| 15:0.5 5:0.3 20:123.2", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryByteByte() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleByteByte() { Dict = new Dictionary() }; example.Dict.Add(15, (Byte)3); example.Dict.Add(5, (Byte)4); example.Dict.Add(20, (Byte)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryByteSByte() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleByteSByte() { Dict = new Dictionary() }; example.Dict.Add(15, (SByte)3); example.Dict.Add(5, (SByte)4); example.Dict.Add(20, (SByte)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryByteInt16() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleByteInt16() { Dict = new Dictionary() }; example.Dict.Add(15, (Int16)3); example.Dict.Add(5, (Int16)4); example.Dict.Add(20, (Int16)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryByteInt32() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleByteInt32() { Dict = new Dictionary() }; example.Dict.Add(15, (Int32)3); example.Dict.Add(5, (Int32)4); example.Dict.Add(20, (Int32)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryByteUInt16() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleByteUInt16() { Dict = new Dictionary() }; example.Dict.Add(15, (UInt16)3); example.Dict.Add(5, (UInt16)4); example.Dict.Add(20, (UInt16)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryByteUInt32() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleByteUInt32() { Dict = new Dictionary() }; example.Dict.Add(15, (UInt32)3); example.Dict.Add(5, (UInt32)4); example.Dict.Add(20, (UInt32)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryByteSingle() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleByteSingle() { Dict = new Dictionary() }; example.Dict.Add(15, (Single)3); example.Dict.Add(5, (Single)4); example.Dict.Add(20, (Single)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryByteInt64() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleByteInt64() { Dict = new Dictionary() }; example.Dict.Add(15, (Int64)3); example.Dict.Add(5, (Int64)4); example.Dict.Add(20, (Int64)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryByteUInt64() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleByteUInt64() { Dict = new Dictionary() }; example.Dict.Add(15, (UInt64)3); example.Dict.Add(5, (UInt64)4); example.Dict.Add(20, (UInt64)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryByteDouble() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleByteDouble() { Dict = new Dictionary() }; example.Dict.Add(15, (Double)3); example.Dict.Add(5, (Double)4); example.Dict.Add(20, (Double)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionarySByte() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleSByte() { Dict = new Dictionary() }; example.Dict.Add(15, .5f); example.Dict.Add(5, .3f); example.Dict.Add(20, 123.2f); vw.Validate("| 15:0.5 5:0.3 20:123.2", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionarySByteString() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleSByteString() { Dict = new Dictionary() }; example.Dict.Add(15, "0.5"); example.Dict.Add(5, "0.3"); example.Dict.Add(20, "123.2"); vw.Validate("| 15:0.5 5:0.3 20:123.2", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionarySByteByte() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleSByteByte() { Dict = new Dictionary() }; example.Dict.Add(15, (Byte)3); example.Dict.Add(5, (Byte)4); example.Dict.Add(20, (Byte)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionarySByteSByte() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleSByteSByte() { Dict = new Dictionary() }; example.Dict.Add(15, (SByte)3); example.Dict.Add(5, (SByte)4); example.Dict.Add(20, (SByte)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionarySByteInt16() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleSByteInt16() { Dict = new Dictionary() }; example.Dict.Add(15, (Int16)3); example.Dict.Add(5, (Int16)4); example.Dict.Add(20, (Int16)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionarySByteInt32() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleSByteInt32() { Dict = new Dictionary() }; example.Dict.Add(15, (Int32)3); example.Dict.Add(5, (Int32)4); example.Dict.Add(20, (Int32)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionarySByteUInt16() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleSByteUInt16() { Dict = new Dictionary() }; example.Dict.Add(15, (UInt16)3); example.Dict.Add(5, (UInt16)4); example.Dict.Add(20, (UInt16)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionarySByteUInt32() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleSByteUInt32() { Dict = new Dictionary() }; example.Dict.Add(15, (UInt32)3); example.Dict.Add(5, (UInt32)4); example.Dict.Add(20, (UInt32)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionarySByteSingle() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleSByteSingle() { Dict = new Dictionary() }; example.Dict.Add(15, (Single)3); example.Dict.Add(5, (Single)4); example.Dict.Add(20, (Single)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionarySByteInt64() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleSByteInt64() { Dict = new Dictionary() }; example.Dict.Add(15, (Int64)3); example.Dict.Add(5, (Int64)4); example.Dict.Add(20, (Int64)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionarySByteUInt64() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleSByteUInt64() { Dict = new Dictionary() }; example.Dict.Add(15, (UInt64)3); example.Dict.Add(5, (UInt64)4); example.Dict.Add(20, (UInt64)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionarySByteDouble() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleSByteDouble() { Dict = new Dictionary() }; example.Dict.Add(15, (Double)3); example.Dict.Add(5, (Double)4); example.Dict.Add(20, (Double)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt16() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt16() { Dict = new Dictionary() }; example.Dict.Add(15, .5f); example.Dict.Add(5, .3f); example.Dict.Add(20, 123.2f); vw.Validate("| 15:0.5 5:0.3 20:123.2", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt16String() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt16String() { Dict = new Dictionary() }; example.Dict.Add(15, "0.5"); example.Dict.Add(5, "0.3"); example.Dict.Add(20, "123.2"); vw.Validate("| 15:0.5 5:0.3 20:123.2", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt16Byte() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt16Byte() { Dict = new Dictionary() }; example.Dict.Add(15, (Byte)3); example.Dict.Add(5, (Byte)4); example.Dict.Add(20, (Byte)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt16SByte() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt16SByte() { Dict = new Dictionary() }; example.Dict.Add(15, (SByte)3); example.Dict.Add(5, (SByte)4); example.Dict.Add(20, (SByte)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt16Int16() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt16Int16() { Dict = new Dictionary() }; example.Dict.Add(15, (Int16)3); example.Dict.Add(5, (Int16)4); example.Dict.Add(20, (Int16)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt16Int32() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt16Int32() { Dict = new Dictionary() }; example.Dict.Add(15, (Int32)3); example.Dict.Add(5, (Int32)4); example.Dict.Add(20, (Int32)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt16UInt16() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt16UInt16() { Dict = new Dictionary() }; example.Dict.Add(15, (UInt16)3); example.Dict.Add(5, (UInt16)4); example.Dict.Add(20, (UInt16)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt16UInt32() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt16UInt32() { Dict = new Dictionary() }; example.Dict.Add(15, (UInt32)3); example.Dict.Add(5, (UInt32)4); example.Dict.Add(20, (UInt32)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt16Single() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt16Single() { Dict = new Dictionary() }; example.Dict.Add(15, (Single)3); example.Dict.Add(5, (Single)4); example.Dict.Add(20, (Single)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt16Int64() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt16Int64() { Dict = new Dictionary() }; example.Dict.Add(15, (Int64)3); example.Dict.Add(5, (Int64)4); example.Dict.Add(20, (Int64)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt16UInt64() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt16UInt64() { Dict = new Dictionary() }; example.Dict.Add(15, (UInt64)3); example.Dict.Add(5, (UInt64)4); example.Dict.Add(20, (UInt64)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt16Double() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt16Double() { Dict = new Dictionary() }; example.Dict.Add(15, (Double)3); example.Dict.Add(5, (Double)4); example.Dict.Add(20, (Double)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt32() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt32() { Dict = new Dictionary() }; example.Dict.Add(15, .5f); example.Dict.Add(5, .3f); example.Dict.Add(20, 123.2f); vw.Validate("| 15:0.5 5:0.3 20:123.2", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt32String() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt32String() { Dict = new Dictionary() }; example.Dict.Add(15, "0.5"); example.Dict.Add(5, "0.3"); example.Dict.Add(20, "123.2"); vw.Validate("| 15:0.5 5:0.3 20:123.2", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt32Byte() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt32Byte() { Dict = new Dictionary() }; example.Dict.Add(15, (Byte)3); example.Dict.Add(5, (Byte)4); example.Dict.Add(20, (Byte)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt32SByte() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt32SByte() { Dict = new Dictionary() }; example.Dict.Add(15, (SByte)3); example.Dict.Add(5, (SByte)4); example.Dict.Add(20, (SByte)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt32Int16() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt32Int16() { Dict = new Dictionary() }; example.Dict.Add(15, (Int16)3); example.Dict.Add(5, (Int16)4); example.Dict.Add(20, (Int16)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt32Int32() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt32Int32() { Dict = new Dictionary() }; example.Dict.Add(15, (Int32)3); example.Dict.Add(5, (Int32)4); example.Dict.Add(20, (Int32)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt32UInt16() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt32UInt16() { Dict = new Dictionary() }; example.Dict.Add(15, (UInt16)3); example.Dict.Add(5, (UInt16)4); example.Dict.Add(20, (UInt16)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt32UInt32() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt32UInt32() { Dict = new Dictionary() }; example.Dict.Add(15, (UInt32)3); example.Dict.Add(5, (UInt32)4); example.Dict.Add(20, (UInt32)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt32Single() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt32Single() { Dict = new Dictionary() }; example.Dict.Add(15, (Single)3); example.Dict.Add(5, (Single)4); example.Dict.Add(20, (Single)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt32Int64() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt32Int64() { Dict = new Dictionary() }; example.Dict.Add(15, (Int64)3); example.Dict.Add(5, (Int64)4); example.Dict.Add(20, (Int64)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt32UInt64() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt32UInt64() { Dict = new Dictionary() }; example.Dict.Add(15, (UInt64)3); example.Dict.Add(5, (UInt64)4); example.Dict.Add(20, (UInt64)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryInt32Double() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleInt32Double() { Dict = new Dictionary() }; example.Dict.Add(15, (Double)3); example.Dict.Add(5, (Double)4); example.Dict.Add(20, (Double)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryUInt16() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleUInt16() { Dict = new Dictionary() }; example.Dict.Add(15, .5f); example.Dict.Add(5, .3f); example.Dict.Add(20, 123.2f); vw.Validate("| 15:0.5 5:0.3 20:123.2", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryUInt16String() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleUInt16String() { Dict = new Dictionary() }; example.Dict.Add(15, "0.5"); example.Dict.Add(5, "0.3"); example.Dict.Add(20, "123.2"); vw.Validate("| 15:0.5 5:0.3 20:123.2", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryUInt16Byte() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleUInt16Byte() { Dict = new Dictionary() }; example.Dict.Add(15, (Byte)3); example.Dict.Add(5, (Byte)4); example.Dict.Add(20, (Byte)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryUInt16SByte() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleUInt16SByte() { Dict = new Dictionary() }; example.Dict.Add(15, (SByte)3); example.Dict.Add(5, (SByte)4); example.Dict.Add(20, (SByte)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryUInt16Int16() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleUInt16Int16() { Dict = new Dictionary() }; example.Dict.Add(15, (Int16)3); example.Dict.Add(5, (Int16)4); example.Dict.Add(20, (Int16)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryUInt16Int32() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleUInt16Int32() { Dict = new Dictionary() }; example.Dict.Add(15, (Int32)3); example.Dict.Add(5, (Int32)4); example.Dict.Add(20, (Int32)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryUInt16UInt16() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleUInt16UInt16() { Dict = new Dictionary() }; example.Dict.Add(15, (UInt16)3); example.Dict.Add(5, (UInt16)4); example.Dict.Add(20, (UInt16)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryUInt16UInt32() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleUInt16UInt32() { Dict = new Dictionary() }; example.Dict.Add(15, (UInt32)3); example.Dict.Add(5, (UInt32)4); example.Dict.Add(20, (UInt32)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryUInt16Single() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleUInt16Single() { Dict = new Dictionary() }; example.Dict.Add(15, (Single)3); example.Dict.Add(5, (Single)4); example.Dict.Add(20, (Single)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryUInt16Int64() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleUInt16Int64() { Dict = new Dictionary() }; example.Dict.Add(15, (Int64)3); example.Dict.Add(5, (Int64)4); example.Dict.Add(20, (Int64)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryUInt16UInt64() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleUInt16UInt64() { Dict = new Dictionary() }; example.Dict.Add(15, (UInt64)3); example.Dict.Add(5, (UInt64)4); example.Dict.Add(20, (UInt64)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionaryUInt16Double() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { var example = new DictionaryExampleUInt16Double() { Dict = new Dictionary() }; example.Dict.Add(15, (Double)3); example.Dict.Add(5, (Double)4); example.Dict.Add(20, (Double)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } } public class NumericExampleByte { [Feature] public System.Byte Value { get; set; } } public class NumericExampleByteArray { [Feature] public System.Byte[] Value { get; set; } } public class NumericExampleByteArrayAnchor { [Feature(AddAnchor = true)] public System.Byte[] Value { get; set; } } public class NumericExampleSByte { [Feature] public System.SByte Value { get; set; } } public class NumericExampleSByteArray { [Feature] public System.SByte[] Value { get; set; } } public class NumericExampleSByteArrayAnchor { [Feature(AddAnchor = true)] public System.SByte[] Value { get; set; } } public class NumericExampleInt16 { [Feature] public System.Int16 Value { get; set; } } public class NumericExampleInt16Array { [Feature] public System.Int16[] Value { get; set; } } public class NumericExampleInt16ArrayAnchor { [Feature(AddAnchor = true)] public System.Int16[] Value { get; set; } } public class NumericExampleInt32 { [Feature] public System.Int32 Value { get; set; } } public class NumericExampleInt32Array { [Feature] public System.Int32[] Value { get; set; } } public class NumericExampleInt32ArrayAnchor { [Feature(AddAnchor = true)] public System.Int32[] Value { get; set; } } public class NumericExampleUInt16 { [Feature] public System.UInt16 Value { get; set; } } public class NumericExampleUInt16Array { [Feature] public System.UInt16[] Value { get; set; } } public class NumericExampleUInt16ArrayAnchor { [Feature(AddAnchor = true)] public System.UInt16[] Value { get; set; } } public class NumericExampleUInt32 { [Feature] public System.UInt32 Value { get; set; } } public class NumericExampleUInt32Array { [Feature] public System.UInt32[] Value { get; set; } } public class NumericExampleUInt32ArrayAnchor { [Feature(AddAnchor = true)] public System.UInt32[] Value { get; set; } } public class NumericExampleSingle { [Feature] public System.Single Value { get; set; } } public class NumericExampleSingleArray { [Feature] public System.Single[] Value { get; set; } } public class NumericExampleSingleArrayAnchor { [Feature(AddAnchor = true)] public System.Single[] Value { get; set; } } public class NumericExampleInt64 { [Feature] public System.Int64 Value { get; set; } } public class NumericExampleInt64Array { [Feature] public System.Int64[] Value { get; set; } } public class NumericExampleInt64ArrayAnchor { [Feature(AddAnchor = true)] public System.Int64[] Value { get; set; } } public class NumericExampleUInt64 { [Feature] public System.UInt64 Value { get; set; } } public class NumericExampleUInt64Array { [Feature] public System.UInt64[] Value { get; set; } } public class NumericExampleUInt64ArrayAnchor { [Feature(AddAnchor = true)] public System.UInt64[] Value { get; set; } } public class NumericExampleDouble { [Feature] public System.Double Value { get; set; } } public class NumericExampleDoubleArray { [Feature] public System.Double[] Value { get; set; } } public class NumericExampleDoubleArrayAnchor { [Feature(AddAnchor = true)] public System.Double[] Value { get; set; } } public class DictionaryExampleByte { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleByteString { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleByteByte { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleByteSByte { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleByteInt16 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleByteInt32 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleByteUInt16 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleByteUInt32 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleByteSingle { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleByteInt64 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleByteUInt64 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleByteDouble { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleSByte { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleSByteString { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleSByteByte { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleSByteSByte { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleSByteInt16 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleSByteInt32 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleSByteUInt16 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleSByteUInt32 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleSByteSingle { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleSByteInt64 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleSByteUInt64 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleSByteDouble { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt16 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt16String { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt16Byte { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt16SByte { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt16Int16 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt16Int32 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt16UInt16 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt16UInt32 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt16Single { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt16Int64 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt16UInt64 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt16Double { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt32 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt32String { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt32Byte { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt32SByte { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt32Int16 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt32Int32 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt32UInt16 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt32UInt32 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt32Single { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt32Int64 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt32UInt64 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleInt32Double { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleUInt16 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleUInt16String { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleUInt16Byte { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleUInt16SByte { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleUInt16Int16 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleUInt16Int32 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleUInt16UInt16 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleUInt16UInt32 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleUInt16Single { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleUInt16Int64 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleUInt16UInt64 { [Feature] public Dictionary Dict { get; set; } } public class DictionaryExampleUInt16Double { [Feature] public Dictionary Dict { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestMarshalNumeric.tt000066400000000000000000000120221323713302100237040ustar00rootroot00000000000000<#@ output extension=".cs" #> using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using VW.Serializer; using VW.Serializer.Attributes; namespace cs_unittest { [TestClass] public class TestMarshalNumeric { <# foreach(var t in new[] { typeof(byte), typeof(sbyte), typeof(Int16), typeof(Int32), typeof(UInt16), typeof(UInt32), typeof(float), typeof(Int64), typeof(UInt64), typeof(double) }) { #> [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumeric<#=t.Name#>() { using(var vw = new VowpalWabbitExampleValidator>(string.Empty)) { vw.Validate("| Value:25", new NumericExample<#=t.Name#>() { Value = 25 }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumeric<#=t.Name#>Array() { using(var vw = new VowpalWabbitExampleValidatorArray>(string.Empty)) { vw.Validate("| :4 :2 :3", new NumericExample<#=t.Name#>Array() { Value = new <#=t#>[] { 4, 2, 3 } }); vw.Validate("", new NumericExample<#=t.Name#>Array()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNumeric<#=t.Name#>ArrayAnchor() { using(var vw = new VowpalWabbitExampleValidatorArrayAnchor>(string.Empty)) { vw.Validate("| :1 :4 :2 :3", new NumericExample<#=t.Name#>ArrayAnchor() { Value = new <#=t#>[] { 4, 2, 3 } }); vw.Validate("", new NumericExample<#=t.Name#>ArrayAnchor()); } } <# } #> <# foreach(var t in new[] { typeof(byte), typeof(sbyte), typeof(Int16), typeof(Int32), typeof(UInt16) }) { #> [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionary<#=t.Name#>() { using(var vw = new VowpalWabbitExampleValidator>(string.Empty)) { var example = new DictionaryExample<#=t.Name#>() { Dict = new Dictionary<<#=t#>, float>() }; example.Dict.Add(15, .5f); example.Dict.Add(5, .3f); example.Dict.Add(20, 123.2f); vw.Validate("| 15:0.5 5:0.3 20:123.2", example); example.Dict = null; vw.Validate("", example); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionary<#=t.Name#>String() { using(var vw = new VowpalWabbitExampleValidatorString>(string.Empty)) { var example = new DictionaryExample<#=t.Name#>String() { Dict = new Dictionary<<#=t#>, string>() }; example.Dict.Add(15, "0.5"); example.Dict.Add(5, "0.3"); example.Dict.Add(20, "123.2"); vw.Validate("| 15:0.5 5:0.3 20:123.2", example); example.Dict = null; vw.Validate("", example); } } <# foreach(var s in new[] { typeof(byte), typeof(sbyte), typeof(Int16), typeof(Int32), typeof(UInt16), typeof(UInt32), typeof(float), typeof(Int64), typeof(UInt64), typeof(double) }) { #> [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionary<#=t.Name#><#=s.Name#>() { using(var vw = new VowpalWabbitExampleValidator<#=s.Name#>>(string.Empty)) { var example = new DictionaryExample<#=t.Name#><#=s.Name#>() { Dict = new Dictionary<<#=t#>, <#=s.Name#>>() }; example.Dict.Add(15, (<#=s.Name#>)3); example.Dict.Add(5, (<#=s.Name#>)4); example.Dict.Add(20, (<#=s.Name#>)5); vw.Validate("| 15:3 5:4 20:5", example); example.Dict = null; vw.Validate("", example); } } <# } #> <# } #> } <# foreach(var t in new[] { typeof(byte), typeof(sbyte), typeof(Int16), typeof(Int32), typeof(UInt16), typeof(UInt32), typeof(float), typeof(Int64), typeof(UInt64), typeof(double) }) { #> public class NumericExample<#=t.Name#> { [Feature] public <#=t#> Value { get; set; } } public class NumericExample<#=t.Name#>Array { [Feature] public <#=t#>[] Value { get; set; } } public class NumericExample<#=t.Name#>ArrayAnchor { [Feature(AddAnchor = true)] public <#=t#>[] Value { get; set; } } <# } #> <# foreach(var t in new[] { typeof(byte), typeof(sbyte), typeof(Int16), typeof(Int32), typeof(UInt16) }) { #> public class DictionaryExample<#=t.Name#> { [Feature] public Dictionary<<#=t#>, float> Dict { get; set; } } public class DictionaryExample<#=t.Name#>String { [Feature] public Dictionary<<#=t#>, String> Dict { get; set; } } <# foreach(var s in new[] { typeof(byte), typeof(sbyte), typeof(Int16), typeof(Int32), typeof(UInt16), typeof(UInt32), typeof(float), typeof(Int64), typeof(UInt64), typeof(double) }) { #> public class DictionaryExample<#=t.Name#><#=s.Name#> { [Feature] public Dictionary<<#=t#>, <#=s#>> Dict { get; set; } } <# } #> <# } #> } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestMarshalling.cs000066400000000000000000000301711323713302100232160ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using VW; using VW.Serializer; using VW.Serializer.Attributes; namespace cs_unittest { [TestClass] public class TestMarshalling { [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestEnumerize() { using(var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| AgeEnumerize25", new ExampleEnum() { AgeEnumerize = 25 }); vw.Validate("| AgeEnumerize0 AgeNumeric:25", new ExampleEnum() { AgeNumeric = 25 }); vw.Validate("| AgeEnumerize0 AgeNumeric:23 AgeEnumChild", new ExampleEnum() { AgeNumeric = 23, AgeEnum = Age.Child }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestString() { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("|abc London", new ExampleString() { Location = "London" }); vw.Validate("", new ExampleString() { }); vw.Validate("", new ExampleString() { Location = "" }); } using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| VideoTitleRich_Homie_Quan_-_\"Blah_Blah_Blah\"___Behind_The_Scenes", new ExampleString4 { Value = "VideoTitleRich Homie Quan - \"Blah Blah Blah\" | Behind The Scenes" }); vw.Validate("| VideoTitleIt's_Official__Your_vibrator_Can_be_Hacked", new ExampleString4 { Value = "VideoTitleIt's Official: Your vibrator Can be Hacked" }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestStringFeatureGroup() { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("|a London", new ExampleString2() { Location = "London" }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestStringNamespace() { try { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("|bc London", new ExampleString3() { Location = "London" }); } Assert.Fail("Expected ArgumentException"); } catch (ArgumentException) { } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestStringEscape() { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| New_York_State", new ExampleStringEscape() { Value = "New York State" }); vw.Validate("| new_York_state", new ExampleStringEscape() { Value = "new York state" }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestStringSplit() { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| New York State", new ExampleStringSplit() { Value = "New York State" }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestStringIncludeName() { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| AgeTeenager", new ExampleStringInclude() { Age = "Teenager" }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestDictionary() { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { var ex = new ExampleDictionary() { Dict = new Dictionary() }; ex.Dict.Add("Age", 25); ex.Dict.Add("Location", 1.2); vw.Validate("| Age:25 Location:1.2", ex); ex.Dict = null; vw.Validate("", ex); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestCustomType() { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| value:2", new ExampleCustomType { Custom = new CustomType { value = 2 } }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestEnumerableString() { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| A New_York B", new ExampleEnumerable { Value = new[] { "A", "New_York", "B" } }); vw.Validate("", new ExampleEnumerable()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestEnumerableKV() { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| A:2 B:3", new ExampleEnumerableKV { Value = new [] { new KeyValuePair("A", 2), new KeyValuePair("B", 3) } }); vw.Validate("", new ExampleEnumerableKV()); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestComplexType() { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("|ootheruser AgeAdult GenderMale PAge25 Views:4321 Boston 6:2.4", new UserContext { User = new UserFeatures { Age = Age.Adult, Gender = Gender.Male, Location = "Boston", PAge = 25, Views = 4321, Dict = new Dictionary { { 6, 2.4f } } } }); vw.Validate("|uuserlda :1 :2 :3", new UserContext { UserLDAVector = new FeatureVector { Vectors = new[] { 2f, 3f } } }); } using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| abc |ddoclda :1 :4 :5", new Document { Id = "abc", LDAVector = new FeatureVector { Vectors = new[] { 4f, 5f } } }); vw.Validate("| abc |ddoclda :1", new Document { Id = "abc", LDAVector = new FeatureVector { Vectors = new float[0] } }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestEnumerizePosition() { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| Position0", new ExampleEnumerize { Position = 0 }); vw.Validate("| Position2", new ExampleEnumerize { Position = 2 }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestBool() { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| OnOff", new ExampleBoolean { OnOff = true }); vw.Validate("| ", new ExampleBoolean { OnOff = false }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestFeatureDiscoveryAll() { using (var vw = new VowpalWabbitExampleValidator(new VowpalWabbitSettings { TypeInspector = TypeInspector.All })) { vw.Validate("| Feature1 Feature2:5", new POCO { Feature1 = true, Feature2 = 5 }); } } } public class POCODict { public Dictionary Features { get; set; } } public class POCO { public bool Feature1 { get; set; } [Feature] public int Feature2 { get; set; } } public class ExampleBoolean { [Feature] public bool OnOff { get; set; } } public class ExampleEnumerize { [Feature(Enumerize = true)] public int Position { get; set; } } public class ExampleStringEscape { [Feature(StringProcessing = StringProcessing.Escape)] public String Value { get; set; } } public class ExampleStringInclude { [Feature(StringProcessing = StringProcessing.EscapeAndIncludeName)] public String Age { get; set; } } public class ExampleStringSplit { [Feature(StringProcessing = StringProcessing.Split)] public String Value { get; set; } } public class UserContext { [Feature(Namespace = "otheruser", FeatureGroup = 'o')] public UserFeatures User { get; set; } [Feature(Namespace = "userlda", FeatureGroup = 'u', AddAnchor = true)] public FeatureVector UserLDAVector { get; set; } public IReadOnlyList ActionDependentFeatures { get; set; } } public class Document { [Feature] public string Id { get; set; } [Feature(Namespace = "doclda", FeatureGroup = 'd', AddAnchor = true)] public FeatureVector LDAVector { get; set; } } public class FeatureVector { [Feature(AddAnchor = true)] public float[] Vectors { get; set; } } public class UserFeatures { [Feature] public Age? Age { get; set; } [Feature(Enumerize = true)] public int? PAge { get; set; } [Feature] public Gender? Gender { get; set; } [Feature] public string Location { get; set; } [Feature] public long Views { get; set; } [Feature] public Dictionary Dict { get; set; } } public enum Gender { Female, Male } public class CustomType { [Feature] public int value { get; set; } } public class ExampleCustomType { [Feature] public CustomType Custom { get; set; } } public class ExampleDictionary { [Feature] public IDictionary Dict { get; set; } } public class ExampleEnumerable { [Feature] public IEnumerable Value { get; set; } } public class ExampleEnumerableKV { [Feature] public IEnumerable> Value { get; set; } } public class ExampleEnum { [Feature(Enumerize = true)] public int AgeEnumerize { get; set; } [Feature] public int? AgeNumeric { get; set; } [Feature] public Age? AgeEnum { get; set; } } public class ExampleString { [Feature(FeatureGroup = 'a', Namespace = "bc")] public string Location { get; set; } } public class ExampleString2 { [Feature(FeatureGroup = 'a')] public string Location { get; set; } } public class ExampleString3 { [Feature(Namespace = "bc")] public string Location { get; set; } } public class ExampleString4 { [Feature(StringProcessing = StringProcessing.Escape)] public string Value { get; set; } } public enum Age { Child, Adult } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestMarshallingOverflow.cs000066400000000000000000000063121323713302100247420ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using System; namespace cs_unittest { [TestClass] public class TestMarshallingOverflow { [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] [TestCategory("Vowpal Wabbit")] public void TestNumericInt64Overflow() { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| Value:9.22337203685477580700E+018", new NumericExampleInt64() { Value = Int64.MaxValue }); vw.Validate("| Value:-9.22337203685477580700E+018", new NumericExampleInt64() { Value = Int64.MinValue}); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] [TestCategory("Vowpal Wabbit")] public void TestNumericUInt64Overflow() { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| Value:1.844674E+19", new NumericExampleUInt64() { Value = UInt64.MaxValue}); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] [TestCategory("Vowpal Wabbit")] public void TestNumericDoubleOverflow() { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| Value:1.79769313486231570000E+308", new NumericExampleDouble() { Value = double.MaxValue }); vw.Validate("| Value:-1.79769313486231570000E+308", new NumericExampleDouble() { Value = double.MinValue }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] [TestCategory("Vowpal Wabbit")] public void TestNumericInt64OverflowArray() { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| 0:9.22337203685477580700E+018", new NumericExampleInt64Array() { Value = new [] { Int64.MaxValue } }); vw.Validate("| 0:-9.22337203685477580700E+018", new NumericExampleInt64Array() { Value = new[] { Int64.MinValue } }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] [TestCategory("Vowpal Wabbit")] public void TestNumericUInt64OverflowArray() { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| 0:1.844674E+19", new NumericExampleUInt64Array() { Value = new[] { UInt64.MaxValue } }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] [TestCategory("Vowpal Wabbit")] public void TestNumericDoubleOverflowArray() { using (var vw = new VowpalWabbitExampleValidator(string.Empty)) { vw.Validate("| 0:1.79769313486231570000E+308", new NumericExampleDoubleArray() { Value = new [] { double.MaxValue } }); vw.Validate("| 0:-1.79769313486231570000E+308", new NumericExampleDoubleArray() { Value = new [] { double.MinValue } }); } } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestModelLoading.cs000066400000000000000000000140021323713302100233060ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Text; using System.Threading.Tasks; using VW; namespace cs_unittest { [TestClass] public class TestModelLoading : TestBase { [TestMethod] [TestCategory("Vowpal Wabbit/Model Loading")] public void TestLoadModelCorrupt() { InternalTestModel(@"model-sets/7.10.2_corrupted.model", false); } [TestMethod] [TestCategory("Vowpal Wabbit/Model Loading")] public void TestLoadModel() { InternalTestModel(@"model-sets/8.0.0_ok.model", true); InternalTestModel(@"model-sets/8.0.1.test_named_ok.model", true); InternalTestModel(@"model-sets/8.0.1_rcv1_ok.model", true); InternalTestModel(@"model-sets/8.0.1_hash_ok.model", true); } [TestMethod] [TestCategory("Vowpal Wabbit/Model Loading")] public void TestLoadModelRandomCorrupt() { InternalTestModelRandomCorrupt("model-sets/8.0.1.test_named_ok.model"); //InternalTestModelRandomCorrupt("model-sets/8.0.1_rcv1_ok.model"); //InternalTestModelRandomCorrupt("model-sets/8.0.1_hash_ok.model"); } [TestMethod] [TestCategory("Vowpal Wabbit/Model Loading")] public void TestLoadModelInMemory() { using (var vw = new VowpalWabbit(@"-i model-sets\8.0.1_rcv1_ok.model")) { var memStream = new MemoryStream(); vw.SaveModel(memStream); vw.SaveModel("native.model"); using (var file = File.Create("managed.file.model")) { vw.SaveModel(file); } var nativeModel = File.ReadAllBytes("native.model"); var managedFileModel = File.ReadAllBytes("managed.file.model"); var managedModel = memStream.ToArray(); Assert.IsTrue(nativeModel.SequenceEqual(managedModel)); Assert.IsTrue(nativeModel.SequenceEqual(managedFileModel)); } } [TestMethod] [TestCategory("Vowpal Wabbit/Model Loading")] public void TestID() { using (var vw = new VowpalWabbit("--id abc")) { Assert.AreEqual("abc", vw.ID); vw.SaveModel("model"); vw.ID = "def"; vw.SaveModel("model.TestID"); } using (var vw = new VowpalWabbit("-i model")) { Assert.AreEqual("abc", vw.ID); } using (var vw = new VowpalWabbit("-i model.TestID")) { Assert.AreEqual("def", vw.ID); } using (var vwm = new VowpalWabbitModel("-i model.TestID")) { Assert.AreEqual("def", vwm.ID); using (var vw = new VowpalWabbit(new VowpalWabbitSettings { Model = vwm })) { Assert.AreEqual("def", vw.ID); Assert.AreEqual(vwm.ID, vw.ID); } } } [TestMethod] [TestCategory("Vowpal Wabbit/Model Loading")] public void TestEmptyID() { using (var vw = new VowpalWabbit("-l 1")) { Assert.AreEqual(string.Empty, vw.ID); vw.SaveModel("model"); } using (var vw = new VowpalWabbit("-f model")) { Assert.AreEqual(string.Empty, vw.ID); } } [TestMethod] [TestCategory("Vowpal Wabbit/Model Loading")] public void TestReload() { using (var vw = new VowpalWabbit("")) { vw.SaveModel("model"); vw.Reload(); } using (var vw = new VowpalWabbit("")) { vw.ID = "def"; vw.SaveModel("model.TestReload"); vw.Reload(); Assert.AreEqual("def", vw.ID); } } private void InternalTestModel(string modelFile, bool shouldPass) { bool passed = false; try { using (var vw = new VowpalWabbitModel(string.Format("--quiet -t -i {0}", modelFile))) { // should only reach this point if model is valid passed = true; } } catch (VowpalWabbitException ex) { Assert.IsTrue(ex.Message.Contains("corrupted")); } if (shouldPass) { Assert.IsTrue(passed); } } private void InternalTestModelRandomCorrupt(string modelFile) { const int numBytesToCorrupt = 10; var rand = new Random(0); byte[] modelBytes = File.ReadAllBytes(modelFile); for (int i = 0; i < 100; i++) { var corruptBytes = new byte[modelBytes.Length]; Array.Copy(modelBytes, corruptBytes, corruptBytes.Length); for (int j = 0; j < numBytesToCorrupt; j++) { corruptBytes[rand.Next(corruptBytes.Length)] = (byte)rand.Next(byte.MaxValue); } try { using (var modelStream = new MemoryStream(corruptBytes)) using (var vw = new VowpalWabbitModel(new VowpalWabbitSettings("--quiet -t") { ModelStream = modelStream })) { // chances of reaching this point after reading a corrupt model are low Assert.IsTrue(false); } } catch (Exception) // an exception should be caught unless AV is encountered in which case the test will fail { Assert.IsTrue(true); } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestMultiClassPredictionFactory.cs000066400000000000000000000023711323713302100264070ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using VW; namespace cs_unittest { [TestClass] public class TestMultiClassPredictionFactory { [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestMultiClassProbabilitiesFactory() { using (var vw = SetupVW()) { var res = vw.Predict(" | a", VowpalWabbitPredictionType.MultiClassProbabilities); Assert.AreEqual(res.Count, 3); Assert.AreEqual(res.Values.Sum(), 1.0f); Assert.IsTrue(res[1] > res[3]); Assert.IsTrue(res[2] > res[3]); var res2 = vw.Predict(" | e", VowpalWabbitPredictionType.MultiClassProbabilities); Assert.IsTrue(res2[3] > res2[1]); Assert.IsTrue(res2[3] > res2[2]); } } private VowpalWabbit SetupVW() { var vw = new VowpalWabbit(" --probabilities --loss_function=logistic --oaa 3"); vw.Learn("1 | a b"); vw.Learn("2 | a c"); vw.Learn("3 | c b e"); return vw; } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestMultiworldTesting.cs000066400000000000000000000030571323713302100244600ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using VW; using VW.Labels; namespace cs_unittest { [TestClass] public class TestMultiworldTestingClass { [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestMultiworldTesting() { using (var mwt = new VowpalWabbitMultiworldTesting()) { AssertAreEqual( mwt.Evaluate(1, 2, new ContextualBanditLabel { Action = 1, Probability = .5f }), 0f, 0f, 0f); AssertAreEqual( mwt.Evaluate(1, 2, new ContextualBanditLabel { Action = 2, Probability = .5f, Cost = 1 }), 0f, 0f, 1f); AssertAreEqual( mwt.Evaluate(1, 2, new ContextualBanditLabel { Action = 1, Probability = .5f }), 0, 0, 0.6666667f); } } private static void AssertAreEqual(VowpalWabbitMultiworldTesting.PoliciesPerformance actual, params float[] expected) { Assert.AreEqual(expected.Length, actual.NumConstantPolicies + 1); Assert.AreEqual(expected[0], actual.LearnedPolicy, 0.0001, "Learned policy differs"); int i = 1; foreach (var value in actual.ConstantPolicies) { Assert.AreEqual(expected[i], value, 0.0001, "Constant policy " + i + " differs"); i++; } } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestNull.cs000066400000000000000000000144471323713302100216770ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using VW; using VW.Labels; using VW.Serializer.Attributes; namespace cs_unittest { [TestClass] public class TestNull { [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNull1() { using (var vw = new VowpalWabbit("--cb_adf --rank_all --interact ab")) { var ctx = new Context() { ID = 25, Vector = null, ActionDependentFeatures = new[] { new ADF { ADFID = "23" } }.ToList() }; vw.Learn(ctx, ctx.ActionDependentFeatures, 0, new ContextualBanditLabel() { Action = 1, Cost = 1, Probability = 0.2f }); var result = vw.Predict(ctx, ctx.ActionDependentFeatures); Assert.AreEqual(1, result.Length); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNull2() { using (var vw = new VowpalWabbit("--cb_adf --rank_all --interact ab")) { var ctx = new Context() { ID = 25, Vector = null, ActionDependentFeatures = new[] { new ADF { ADFID = "23", } }.ToList() }; vw.Learn(ctx, ctx.ActionDependentFeatures, 0, new ContextualBanditLabel() { Action = 1, Cost= 1, Probability = 0.2f }); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNull3() { using (var vw = new VowpalWabbit("--cb_adf --rank_all --interact ac")) { var ctx = new Context() { ID = 25, Vector = new float[] { 3 }, VectorC = new float[] { 2, 2, 3 }, ActionDependentFeatures = new[] { new ADF { ADFID = "23", } }.ToList() }; var label = new ContextualBanditLabel() { Action = 1, Cost= 1, Probability = 0.2f }; vw.Learn(ctx, ctx.ActionDependentFeatures, 0, label); ctx.Vector = null; vw.Learn(ctx, ctx.ActionDependentFeatures, 0, label); ctx.Vector = new float[] { 2 }; ctx.VectorC = null; vw.Learn(ctx, ctx.ActionDependentFeatures, 0, label); ctx.Vector = null; vw.Learn(ctx, ctx.ActionDependentFeatures, 0, label); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNull4() { using (var vw = new VowpalWabbit("--cb_adf --rank_all --interact ab")) { var ctx = new Context() { ID = 25, Vector = null, ActionDependentFeatures = new[] { new ADF { ADFID = null } }.ToList() }; var label = new ContextualBanditLabel() { Action = 1, Cost= 1, Probability = 0.2f }; vw.Learn(ctx, ctx.ActionDependentFeatures, 0, label); var result = vw.Predict(ctx, ctx.ActionDependentFeatures); Assert.AreEqual(1, result.Length); ctx.ID = null; vw.Learn(ctx, ctx.ActionDependentFeatures, 0, label); result = vw.Predict(ctx, ctx.ActionDependentFeatures); Assert.AreEqual(1, result.Length); } } [TestMethod] [TestCategory("Vowpal Wabbit/Marshal")] public void TestNull5() { using (var vw = new VowpalWabbit("--cb_adf --rank_all --interact ab")) { var ctx = new Context() { ID = 25, ActionDependentFeatures = new[] { new ADF { ADFID = "123" }, new ADF(), new ADF(), new ADF { ADFID = "4"} }.ToList() }; var label = new ContextualBanditLabel() { Action = 1, Cost = 1, Probability = 0.2f }; vw.Learn(ctx, ctx.ActionDependentFeatures, 0, label); var result = vw.Predict(ctx, ctx.ActionDependentFeatures); Assert.AreEqual(4, result.Length); ctx.ActionDependentFeatures[0].ADFID = null; ctx.ActionDependentFeatures[3].ADFID = null; result = vw.Predict(ctx, ctx.ActionDependentFeatures); Assert.AreEqual(4, result.Length); } } } public class ADF { [Feature] public string ADFID { get; set; } [Feature(FeatureGroup = 'b', AddAnchor = true)] public float[] Vector { get; set; } public ILabel Label { get; set; } } public class Context { [Feature] public int? ID { get; set; } [Feature(FeatureGroup = 'a', AddAnchor = true)] public float[] Vector { get; set; } [Feature(FeatureGroup = 'c')] public float[] VectorC { get; set; } public IReadOnlyList ActionDependentFeatures { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestPooling.cs000066400000000000000000000162541323713302100223720ustar00rootroot00000000000000using System; using System.Collections.Generic; using System.Threading; using Microsoft.VisualStudio.TestTools.UnitTesting; using VW; using VW.Labels; using VW.Serializer.Attributes; using System.IO; namespace cs_unittest { [TestClass] public class TestPooling { [TestMethod] [ExpectedException(typeof(InvalidOperationException))] [TestCategory("Vowpal Wabbit")] public void ObjectPoolTestEmptyFactory() { new ObjectPool().GetOrCreate(); } [TestMethod] [ExpectedException(typeof(ObjectDisposedException))] [TestCategory("Vowpal Wabbit")] public void ObjectPoolTestDisposed1() { var objectPool = new ObjectPool(ObjectFactory.Create(new Disposable(), d => d.Create())); objectPool.Dispose(); objectPool.GetOrCreate(); } [TestMethod] [ExpectedException(typeof(ObjectDisposedException))] [TestCategory("Vowpal Wabbit")] public void ObjectPoolTestDisposed2() { var objectPool = new ObjectPool(ObjectFactory.Create(new Disposable(), d => d.Create())); objectPool.Dispose(); objectPool.UpdateFactory(ObjectFactory.Create(new Disposable(), d => d.Create())); } [TestMethod] [TestCategory("Vowpal Wabbit")] public void ObjectPoolTestDangling() { var factory = new Disposable(); var objectPool = new ObjectPool(ObjectFactory.Create(factory, d => d.Create())); var p1 = objectPool.GetOrCreate(); objectPool.Dispose(); Assert.IsTrue(factory.Disposed); Assert.AreEqual(1, factory.Children.Count); Assert.IsFalse(factory.Children[0].Disposed); // don't throw exception if we return pool too late p1.Dispose(); } [TestMethod] [TestCategory("Vowpal Wabbit")] public void ObjectPoolTestFactory() { var factory1 = new Disposable(); var factory2 = new Disposable(); var objectPool = new ObjectPool(); objectPool.UpdateFactory(ObjectFactory.Create(factory1, d => d.Create())); var p3 = objectPool.GetOrCreate(); p3.Dispose(); objectPool.UpdateFactory(ObjectFactory.Create(factory2, d => d.Create())); Assert.IsTrue(factory1.Disposed); var p1 = objectPool.GetOrCreate(); var p2 = objectPool.GetOrCreate(); p1.Dispose(); p2.Dispose(); p1 = objectPool.GetOrCreate(); p1.Dispose(); objectPool.Dispose(); factory1.AssertChildrenDisposed(); Assert.AreEqual(1, factory1.Children.Count); factory2.AssertChildrenDisposed(); Assert.AreEqual(2, factory2.Children.Count); Assert.IsTrue(factory2.Disposed); } [TestMethod] [TestCategory("Vowpal Wabbit")] public void ThreadPoolNull() { using (var pool = new VowpalWabbitThreadedPrediction()) { using (var vw = pool.GetOrCreate()) { Assert.IsNull(vw.Value); } pool.UpdateModel(new VowpalWabbitModel(string.Empty)); using (var vw = pool.GetOrCreate()) { Assert.IsNotNull(vw.Value); } } } [TestMethod] [TestCategory("Vowpal Wabbit")] public void ObjectPoolTestConcurrency() { var factories = new List { new Disposable() }; var objectPool = new ObjectPool(ObjectFactory.Create(factories[0], d => d.Create())); var t1 = new Thread(() => { for (int i = 0; i < 500; i++) { var p = objectPool.GetOrCreate(); Thread.Sleep(5); p.Dispose(); } }); var t2 = new Thread(() => { for (int i = 0; i < 500; i++) { var p = objectPool.GetOrCreate(); Thread.Sleep(7); p.Dispose(); } }); var t3 = new Thread(() => { for (int i = 0; i < 20; i++) { var f = new Disposable(); objectPool.UpdateFactory(ObjectFactory.Create(f, d => d.Create())); Thread.Sleep(20); } }); t1.Start(); t2.Start(); t3.Start(); t1.Join(); t2.Join(); t3.Join(); objectPool.Dispose(); foreach (var f in factories) { Assert.IsTrue(f.Disposed); f.AssertChildrenDisposed(); } } public class SimpleData { [Feature] public float Value { get; set; } } [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestSaveLoadSkip() { using (var vw = new VowpalWabbit("--binary -f saveload.model")) { for (int i = 0; i < 100; i++) { vw.Learn(new SimpleData { Value = 1 }, new SimpleLabel { Label = 1 }); vw.Learn(new SimpleData { Value = -1 }, new SimpleLabel { Label = -1 }); } Assert.AreEqual(1, vw.Predict(new SimpleData { Value = 1 }, VowpalWabbitPredictionType.Scalar)); Assert.AreEqual(-1, vw.Predict(new SimpleData { Value = -1 }, VowpalWabbitPredictionType.Scalar)); } using (var model = new VowpalWabbitModel(new VowpalWabbitSettings { Arguments = "--binary", ModelStream = File.Open("saveload.model", FileMode.Open) })) using (var pool = new VowpalWabbitThreadedPrediction(new VowpalWabbitSettings { Model = model })) { using (var vw = pool.GetOrCreate()) { Assert.AreEqual(-1, vw.Value.Predict(new SimpleData { Value = -1 }, VowpalWabbitPredictionType.Scalar)); } } } public class Disposable : IDisposable { public Disposable() { this.Children = new List(); } public void AssertChildrenDisposed() { foreach (var item in this.Children) { Assert.IsTrue(item.Disposed); } } public List Children { get; set; } public bool Disposed { get; set; } public void Dispose() { this.Disposed = true; } public Disposable Create() { var d = new Disposable(); this.Children.Add(d); return d; } } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestRunner.cs000066400000000000000000000024511323713302100222260ustar00rootroot00000000000000using cs_testcommon; using System; using System.Reflection; namespace cs_unittest { /// /// Helper to perform leak testing /// public class TestRunner : MarshalByRefObject, ITestRunner { public string Run(string type, string method) { var testType = Type.GetType(type); var testObject = Activator.CreateInstance(testType); try { var m = testType.GetMethod(method); if (m == null) { return string.Format("TestRunner: {0}.{1} not found", type, method); } m.Invoke(testObject, null); } catch (Exception ex) { var tex = ex as TargetInvocationException; if (tex != null) { ex = tex.InnerException; } return string.Format("{0}\n{1}\n#-#-#-#-#-#-#{2}", ex.GetType(), ex.Message, ex.StackTrace); } finally { var disposable = testObject as IDisposable; if (disposable != null) { disposable.Dispose(); } } return null; } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestSerializer.cs000066400000000000000000000104401323713302100230630ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections.Generic; using System.Linq; using System.Linq.Expressions; using System.Reflection; using System.Text; using System.Threading.Tasks; using VW; using VW.Reflection; using VW.Serializer; using VW.Serializer.Attributes; using VW.Serializer.Intermediate; namespace cs_unittest { public class CustomClass { public int X { get; set; } public bool HasVisited = false; } public class MyContext { [Feature] public CustomClass Feature { get; set; } } public class CustomFeaturizer { public void MarshalFeature(VowpalWabbitMarshalContext context, Namespace ns, Feature feature, CustomClass value) { Assert.IsNotNull(context); Assert.IsNotNull(ns); Assert.IsNotNull(feature); Assert.IsNotNull(value); Assert.AreEqual(5, value.X); value.HasVisited = true; } } public class MyDictifyContext { [Feature] public int A { get; set; } [Feature(Dictify = true)] public float[] B { get; set; } } [TestClass] public class TestSerializer { [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestCustomFeaturizer() { var context = new MyContext() { Feature = new CustomClass() { X = 5 }}; using (var vw = new VowpalWabbit("")) { var serializer = VowpalWabbitSerializerFactory.CreateSerializer(new VowpalWabbitSettings { CustomFeaturizer = new List { typeof(CustomFeaturizer) } }) .Create(vw); var example = serializer.Serialize(context); Assert.IsNotNull(example); example.Dispose(); } Assert.IsTrue(context.Feature.HasVisited); } [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestCustomFeaturizerOverideMethod() { var context = new MyContext() { Feature = new CustomClass() { X = 5 } }; using (var vw = new VowpalWabbit("")) { var serializer = VowpalWabbitSerializerFactory.CreateSerializer(new VowpalWabbitSettings { Schema = new Schema { Features = new List { new FeatureExpression(typeof(CustomClass), "Feature", // TODO: looks a bit awkward for an API. The compiler needs to know what property to access to copy the value into the Feature object valueExpression => Expression.Property(valueExpression, (PropertyInfo)ReflectionHelper.GetInfo((MyContext m) => m.Feature)), overrideSerializeMethod: (MethodInfo)ReflectionHelper.GetInfo((CustomFeaturizer c) => c.MarshalFeature(null, null, null, null))) } } }).Create(vw); var example = serializer.Serialize(context); Assert.IsNotNull(example); example.Dispose(); } Assert.IsTrue(context.Feature.HasVisited); } [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestDictify() { using (var vw = new VowpalWabbit(new VowpalWabbitSettings { EnableStringExampleGeneration = true, EnableStringFloatCompact = true })) using (var serializer = VowpalWabbitSerializerFactory.CreateSerializer(vw.Settings).Create(vw)) { var dictionary = new Dictionary(); var ctx = new MyDictifyContext { A = 5, B = new[] { 1f, 2f, 3f } }; var str = serializer.SerializeToString(ctx, dictionary: dictionary); Assert.AreEqual(" | A:5 d0", str); Assert.AreEqual(1, dictionary.Count); Assert.IsTrue(dictionary.ContainsKey(" 0:1 1:2 2:3")); Assert.AreEqual("d0", dictionary[" 0:1 1:2 2:3"]); } } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestTelemetryProcessor.cs000066400000000000000000000027261323713302100246340ustar00rootroot00000000000000using Microsoft.ApplicationInsights.Channel; using Microsoft.ApplicationInsights.DataContracts; using Microsoft.ApplicationInsights.Extensibility; using Microsoft.ApplicationInsights.Extensibility.Implementation; using System; using System.Collections.Generic; using System.Diagnostics; using System.Linq; using System.Text; using System.Threading.Tasks; namespace cs_unittest { public class TestTelemetryProcessor: ITelemetryProcessor { private ITelemetryProcessor Next { get; set; } public SynchronizedCollection Exceptions { get; private set; } // Link processors to each other in a chain. public TestTelemetryProcessor(ITelemetryProcessor next, SynchronizedCollection exceptions) { this.Next = next; this.Exceptions = exceptions; } public void Process(ITelemetry item) { var tt = item as TraceTelemetry; if (tt != null) { Console.WriteLine($"Trace: {tt.Message}"); foreach (var prop in tt.Properties) Console.WriteLine($"\t{prop.Key}: {prop.Value}"); } var et = item as ExceptionTelemetry; if (et != null) { this.Exceptions.Add(et); Console.WriteLine($"Exception: {et.Message}. {et.Exception.StackTrace}"); } this.Next.Process(item); } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestTracing.cs000066400000000000000000000016731323713302100223510ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using VW; namespace cs_unittest { [TestClass] public class TestTracing { [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestTraceListener() { var messages = new List(); using (var vw = new VowpalWabbit(new VowpalWabbitSettings { TraceListener = msg => messages.Add(msg), Verbose = true })) { vw.Learn("1 |a x:2"); vw.Learn("2 |a x:3"); } var trace = string.Join("\n", messages); Assert.AreEqual(16, messages.Count, $"Expected 16 lines. Found {messages.Count}. '{trace}'"); Assert.AreEqual("total feature number = 4", messages[15]); } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestWiki.cs000066400000000000000000000036271323713302100216660ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using VW; using VW.Labels; namespace cs_unittest { [TestClass] public class TestWikiClass { [TestMethod] [TestCategory("Vowpal Wabbit")] public void TestWiki() { using (var vw = new VW.VowpalWabbit("-f rcv1.model")) { // 1 |f 13:3.9656971e-02 24:3.4781646e-02 69:4.6296168e-02 using (var exampleBuilder = new VW.VowpalWabbitExampleBuilder(vw)) { // important to dispose the namespace builder at the end, as data is only added to the example // if there is any feature added to the namespace using (var ns = exampleBuilder.AddNamespace('f')) { var namespaceHash = vw.HashSpace("f"); var featureHash = vw.HashFeature("13", namespaceHash); ns.AddFeature(featureHash, 8.5609287e-02f); featureHash = vw.HashFeature("24", namespaceHash); ns.AddFeature(featureHash, 3.4781646e-02f); featureHash = vw.HashFeature("69", namespaceHash); ns.AddFeature(featureHash, 4.6296168e-02f); } exampleBuilder.ApplyLabel(new SimpleLabel() { Label = 1 }); // hand over of memory management using (var example = exampleBuilder.CreateExample()) { VowpalWabbitExampleValidator.Validate("1 |f 13:8.5609287e-02 24:3.4781646e-02 69:4.6296168e-02", example, VowpalWabbitLabelComparator.Simple); vw.Learn(example); } } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TestWrapper.cs000066400000000000000000000030121323713302100223670ustar00rootroot00000000000000using System.IO; using cs_test; using Microsoft.VisualStudio.TestTools.UnitTesting; using VW; namespace cs_unittest { [TestClass] public class TestWrapper : TestBase { [TestMethod] [TestCategory("Vowpal Wabbit")] public void VwCleanupTest() { new VowpalWabbit("-k -l 20 --initial_t 128000 --power_t 1 -c --cache_file VwCleanupTest.cache --passes 8 --invariant --ngram 3 --skips 1 --holdout_off") .Dispose(); } [TestMethod] [TestCategory("Vowpal Wabbit")] public void VwCleanupTestError() { try { if (Directory.Exists("models_out")) Directory.Delete("models_out", true); var vw = new VowpalWabbit("-k -l 20 --initial_t 128000 --power_t 1 -f models_out/0001.model -c --passes 8 --invariant --ngram 3 --skips 1 --holdout_off"); vw.Dispose(); Assert.Fail("Excepted exception not thrown"); } catch (VowpalWabbitException e) { Assert.IsFalse(string.IsNullOrEmpty(e.Filename)); Assert.AreNotEqual(0, e.LineNumber); Assert.IsTrue(e.Message.Contains("No such file or directory"), e.Message); } } [TestMethod] [TestCategory("Vowpal Wabbit")] public void VwModelRefCountingTest() { var model = new VowpalWabbitModel(""); model.Dispose(); } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TrainSet0002Dat.cs000066400000000000000000000042071323713302100226020ustar00rootroot00000000000000using System.Collections.Generic; using System.Globalization; using cs_unittest; using VW.Labels; using VW.Serializer.Attributes; namespace TrainSet0002Dat { public class Data : BaseData { [Feature(FeatureGroup = 'T', Name = "")] public string T { get; set; } [Feature(FeatureGroup = 'f')] public List> F { get; set; } public ILabel Label { get; set; } } public class DataListener : VowpalWabbitListenerToEvents { private Data example; public override void EnterExample(VowpalWabbitParser.ExampleContext context) { this.example = new Data() { F = new List>() }; } public override void ExitExample(VowpalWabbitParser.ExampleContext context) { this.example.Line = context.GetText(); this.Created(this.example.Line, this.example, this.example.Label); } public override void ExitLabel_simple(VowpalWabbitParser.Label_simpleContext context) { var simpleLabel = new SimpleLabel() { Label = context.value.value }; if (context.weight != null) simpleLabel.Weight = context.weight.value; this.example.Label = simpleLabel; } public override void ExitNumber(VowpalWabbitParser.NumberContext context) { context.value = float.Parse(context.GetText(), CultureInfo.InvariantCulture); } public override void ExitFeatureSparse(VowpalWabbitParser.FeatureSparseContext context) { var index = context.index; var weight_index = index.Text; var x = context.x; if (x == null) { // hashed feature this.example.T = weight_index; } else { // sparse feature this.example.F.Add(new KeyValuePair(weight_index, context.x.value)); } } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/TrainSetCs_testLdf.cs000066400000000000000000000053561323713302100236300ustar00rootroot00000000000000using System.Collections.Generic; using VW; using VW.Labels; using VW.Serializer.Attributes; namespace cs_unittest { public static class TrainSetCs_testLdf { public static Cs_TestData[] CreateSampleCbAdfData() { var sampleData = new Cs_TestData[3]; //shared | s_1 s_2 //0:1.0:0.5 | a_1 b_1 c_1 //| a_2 b_2 c_2 //| a_3 b_3 c_3 //| b_1 c_1 d_1 //0:0.0:0.5 | b_2 c_2 d_2 //| a_1 b_1 c_1 //| a_3 b_3 c_3 sampleData[0] = new Cs_TestData { Shared = new[] { "s_1", "s_2" }, ActionDependentFeatures = new[] { new Cs_TestCs_TestDataADF { Features = new[] { "a_1", "b_1", "c_1" }, Label = new ContextualBanditLabel { Cost = 1f, Probability = .5f } }, new Cs_TestCs_TestDataADF { Features = new [] { "a_2","b_2","c_2" } }, new Cs_TestCs_TestDataADF { Features = new [] { "a_3","b_3","c_3" } }, } }; sampleData[1] = new Cs_TestData { ActionDependentFeatures = new[] { new Cs_TestCs_TestDataADF { Features = new [] { "b_1","c_1","d_1" } }, new Cs_TestCs_TestDataADF { Features = new [] { "b_2", "c_2", "d_2" }, Label = new ContextualBanditLabel { Cost = 0f, Probability = .5f } }, } }; sampleData[2] = new Cs_TestData { ActionDependentFeatures = new[] { new Cs_TestCs_TestDataADF { Features = new [] { "a_1","b_1","c_1" } }, new Cs_TestCs_TestDataADF { Features = new [] { "a_3","b_3","c_3" } } } }; return sampleData; } } public class Cs_TestData { [Feature] public string[] Shared { get; set; } public IReadOnlyList ActionDependentFeatures { get; set; } } public class Cs_TestCs_TestDataADF { [Feature] public string[] Features { get; set; } public override string ToString() { return string.Join(" ", this.Features); } public ILabel Label { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/VWTestHelper.cs000066400000000000000000000262761323713302100224640ustar00rootroot00000000000000using System; using System.Globalization; using System.IO; using System.Linq; using Antlr4.Runtime; using Antlr4.Runtime.Atn; using Antlr4.Runtime.Tree; using Microsoft.VisualStudio.TestTools.UnitTesting; using VW; using VW.Serializer; namespace cs_unittest { internal static class VWTestHelper { internal static void ParseInput(string text, IParseTreeListener listener) { ParseInput(new AntlrInputStream(text), listener); } internal static void ParseInput(Stream stream, IParseTreeListener listener) { ParseInput(new UnbufferedCharStream(stream), listener); } internal static void ParseInput(ICharStream stream, IParseTreeListener listener) { // optimized for memory consumption var lexer = new VowpalWabbitLexer(stream) { TokenFactory = new CommonTokenFactory(copyText: true) }; var tokens = new UnbufferedTokenStream(lexer); var parser = new VowpalWabbitParser(tokens) { // Note; don't disable, as it is required to access the line // BuildParseTree = false, }; // fast than LL(*) parser.Interpreter.PredictionMode = PredictionMode.Sll; parser.AddParseListener(listener); parser.AddErrorListener(new TestErrorListener()); parser.start(); } internal static void Learn(string args, string inputFile, string stderrFile) where TListener : VowpalWabbitListenerToEvents, new() { using (var vw = new VowpalWabbit(args)) using (var validate = new VowpalWabbitExampleValidator(args)) { var listener = new TListener(); listener.Created = (line, data, label) => { if (data == null) { Assert.Fail("got empty example"); } validate.Validate(line, data, label); vw.Learn(data, label); }; VWTestHelper.ParseInput(File.OpenRead(inputFile), listener); AssertEqual(stderrFile, vw.Native.PerformanceStatistics); } } internal static void Predict(string args, string inputFile, string referenceFile = null) where TData : BaseData where TListener : VowpalWabbitListenerToEvents, new() { float[] references = null; var index = 0; if (referenceFile != null) { references = File.ReadAllLines(referenceFile) .Select(l => float.Parse(l.Split(' ')[0], CultureInfo.InvariantCulture)) .ToArray(); } using (var vwRef = new VowpalWabbit(args)) using (var vwModel = new VowpalWabbitModel(args)) using (var vwValidate = new VowpalWabbit(args)) using (var vwInMemoryShared2 = new VowpalWabbit(new VowpalWabbitSettings { Model = vwModel })) using (var validate = new VowpalWabbitExampleValidator(args)) { var listener = new TListener(); listener.Created = (line, x, label) => { validate.Validate(line, x, label); var expectedDynamic = vwRef.Predict(x.Line, VowpalWabbitPredictionType.Dynamic); Assert.IsInstanceOfType(expectedDynamic, typeof(float)); var expected = vwRef.Predict(x.Line, VowpalWabbitPredictionType.Scalar); var actual = vwInMemoryShared2.Predict(x, VowpalWabbitPredictionType.Scalar, label); Assert.AreEqual((float)expectedDynamic, actual, 1e-5); Assert.AreEqual(expected, actual, 1e-5); if (references != null) Assert.AreEqual(references[index++], actual, 1e-5); }; } } internal static void AssertEqual(string expectedFile, VowpalWabbitPerformanceStatistics actual) { var expectedPerformanceStatistics = ReadPerformanceStatistics(expectedFile); AssertEqual(expectedPerformanceStatistics, actual); } internal static void FuzzyEqual(double? expected, double actual, double epsilon, string message) { if (expected == null) return; // from test/RunTests var delta = Math.Abs(expected.Value - actual); if (delta > epsilon) { // We have a 'big enough' difference, but this difference // may still not be meaningful in all contexts: // Big numbers should be compared by ratio rather than // by difference // Must ensure we can divide (avoid div-by-0) if (Math.Abs(actual) <= 1.0) { // If numbers are so small (close to zero), // ($delta > $Epsilon) suffices for deciding that // the numbers are meaningfully different Assert.Fail(string.Format("{0} vs {1}: delta={2} > Epsilon={3}: {4}", expected, actual, delta, epsilon, message)); } // Now we can safely divide (since abs($word2) > 0) // and determine the ratio difference from 1.0 var ratio_delta = Math.Abs(expected.Value / actual - 1.0); if (ratio_delta > epsilon) { Assert.Fail(string.Format("{0} vs {1}: delta={2} > Epsilon={3}: {4}", expected, actual, delta, epsilon, message)); } } } internal static void AssertEqual(VowpalWabbitPerformanceStatistics expected, VowpalWabbitPerformanceStatistics actual) { if (expected.TotalNumberOfFeatures != actual.TotalNumberOfFeatures) { Console.Error.WriteLine( "Warning: total number of features differs. Expected: {0} vs. actual: {1}", expected.TotalNumberOfFeatures, actual.TotalNumberOfFeatures); } Assert.AreEqual(expected.NumberOfExamplesPerPass, actual.NumberOfExamplesPerPass, "NumberOfExamplesPerPass"); FuzzyEqual(expected.AverageLoss, actual.AverageLoss, 1e-3, "AverageLoss"); FuzzyEqual(expected.BestConstant, actual.BestConstant, 1e-3, "BestConstant"); // TODO: something weir'd is happening here. BestConstantsLoss is 0 if using RunAll // has the proper value if just the unit test is run //Console.WriteLine(expected.BestConstantLoss + " vs. " + actual.BestConstantLoss); //Assert.AreEqual(expected.BestConstantLoss, actual.BestConstantLoss, 1e-5); FuzzyEqual(expected.WeightedExampleSum, actual.WeightedExampleSum, 1e-3, "WeightedExampleSum"); FuzzyEqual(expected.WeightedLabelSum, actual.WeightedLabelSum, 1e-3, "WeightedLabelSum"); } internal static void AssertEqual(VowpalWabbitStdErrPerformanceStatistics expected, VowpalWabbitPerformanceStatistics actual) { if (expected.TotalNumberOfFeatures != actual.TotalNumberOfFeatures) { Console.Error.WriteLine( "Warning: total number of features differs. Expected: {0} vs. actual: {1}", expected.TotalNumberOfFeatures, actual.TotalNumberOfFeatures); } if (expected.NumberOfExamplesPerPass != null) Assert.AreEqual(expected.NumberOfExamplesPerPass, actual.NumberOfExamplesPerPass, "NumberOfExamplesPerPass"); FuzzyEqual(expected.AverageLoss, actual.AverageLoss, 1e-3, "AverageLoss"); FuzzyEqual(expected.BestConstant, actual.BestConstant, 1e-3, "BestConstant"); // TODO: something weir'd is happening here. BestConstantsLoss is 0 if using RunAll // has the proper value if just the unit test is run //Console.WriteLine(expected.BestConstantLoss + " vs. " + actual.BestConstantLoss); //Assert.AreEqual(expected.BestConstantLoss, actual.BestConstantLoss, 1e-5); FuzzyEqual(expected.WeightedExampleSum, actual.WeightedExampleSum, 1e-3, "WeightedExampleSum"); FuzzyEqual(expected.WeightedLabelSum, actual.WeightedLabelSum, 1e-3, "WeightedLabelSum"); } internal static VowpalWabbitStdErrPerformanceStatistics ReadPerformanceStatistics(string filename) { var lines = File.ReadAllLines(filename); var numExamples = FindULongEntry(lines, "number of examples per pass = "); if (numExamples == 0) numExamples = FindULongEntry(lines, "number of examples = "); var stats = new VowpalWabbitStdErrPerformanceStatistics() { NumberOfExamplesPerPass = numExamples, TotalNumberOfFeatures = FindULongEntry(lines, "total feature number = "), AverageLoss = FindAverageLossEntry(lines), BestConstant = FindDoubleEntry(lines, "best constant = "), BestConstantLoss = FindDoubleEntry(lines, "best constant's loss = "), WeightedExampleSum = FindDoubleEntry(lines, "weighted example sum = "), WeightedLabelSum = FindDoubleEntry(lines, "weighted label sum = ") }; return stats; } private static double? FindAverageLossEntry(string[] lines) { var label = "average loss = "; var candidate = lines.FirstOrDefault(l => l.StartsWith(label)); if (candidate == null) { return null; } candidate = candidate.Substring(label.Length); if (candidate.EndsWith(" h")) { candidate = candidate.Substring(0, candidate.Length - 2); } var ret = 0.0; if (double.TryParse(candidate, NumberStyles.Float, CultureInfo.InvariantCulture, out ret)) { return ret; } return null; } private static double? FindDoubleEntry(string[] lines, string label) { var candidate = lines.FirstOrDefault(l => l.StartsWith(label)); if (candidate == null) { return null; } var ret = 0.0; if (double.TryParse(candidate.Substring(label.Length), NumberStyles.Float, CultureInfo.InvariantCulture, out ret)) { return ret; } return null; } private static ulong? FindULongEntry(string[] lines, string label) { var candidate = lines.FirstOrDefault(l => l.StartsWith(label)); if (candidate == null) { return null; } ulong ret = 0L; if (ulong.TryParse(candidate.Substring(label.Length), NumberStyles.Float, CultureInfo.InvariantCulture, out ret)) { return ret; } return null; } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/VowpalWabbit.g4000066400000000000000000000013611323713302100224220ustar00rootroot00000000000000grammar VowpalWabbit; number returns [float value] : NUMBER; label_simple : value=number (WS weight=number)? WS; label_cbf : value=INT ':' weight=NUMBER; namespace : '|' name=STRING? (WS feature)+ WS?; feature : index=(STRING | NUMBER) (':' x=number)? # FeatureSparse | ':' x=NUMBER # FeatureDense ; // needs more testing tag : ('`' STRING)? WS | STRING ; example : label_simple tag? namespace (WS namespace)*; start : (example NEWLINE)* example (NEWLINE | EOF); // greedy matching, if same length its matched in order NUMBER : INT | FLOAT; INT : [+-]? [0-9]+ ([Ee] '-'? [0-9]+)?; FLOAT : [+-]? [0-9]* '.' [0-9]+ ([Ee] '-'? [0-9]+)?; WS : [ \t]+; NEWLINE : '\r'? '\n'; STRING : ~([:| \t\r\n])+; vowpal-wabbit-8.5.0.dfsg1/cs/unittest/VowpalWabbitExampleJsonValidator.cs000066400000000000000000000232761323713302100265420ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using Newtonsoft.Json; using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Text; using System.Threading.Tasks; using VW; using VW.Labels; using VW.Serializer; namespace cs_unittest { internal sealed class VowpalWabbitExampleJsonValidator : IDisposable { private VowpalWabbit vw; internal VowpalWabbitExampleJsonValidator(string args = null) : this(new VowpalWabbitSettings((args ?? "") + " --json")) { } internal VowpalWabbitExampleJsonValidator(VowpalWabbitSettings settings) { settings = (VowpalWabbitSettings)settings.Clone(); settings.EnableStringExampleGeneration = true; this.vw = new VowpalWabbit(settings); } public void Validate(string line, VowpalWabbitExampleCollection example, IVowpalWabbitLabelComparator labelComparator = null, ILabel label = null, bool validateVowpalWabbitString = true) { Assert.IsNotNull(example); var jsonExample = example as VowpalWabbitSingleLineExampleCollection; Assert.IsNotNull(jsonExample); using (var strExample = this.vw.ParseLine(line)) { var diff = strExample.Diff(this.vw, jsonExample.Example, labelComparator); Assert.IsNull(diff, diff + " generated string: '" + jsonExample.VowpalWabbitString + "'"); if (validateVowpalWabbitString) { using (var strJsonExample = this.vw.ParseLine(jsonExample.Example.VowpalWabbitString)) { diff = strExample.Diff(this.vw, strJsonExample, labelComparator); Assert.IsNull(diff, diff); } } } } public void Validate(string line, string json, IVowpalWabbitLabelComparator labelComparator = null, ILabel label = null, bool enableNativeJsonValidation = true) { using (var jsonSerializer = new VowpalWabbitJsonSerializer(this.vw)) using (var jsonExample = jsonSerializer.ParseAndCreate(json, label)) { this.Validate(line, jsonExample, labelComparator, label); if (enableNativeJsonValidation) { var examples = this.vw.ParseJson(json); Assert.AreEqual(1, examples.Count); using (var jsonNativeExample = new VowpalWabbitSingleLineExampleCollection(this.vw, examples[0])) { this.Validate(line, jsonNativeExample, labelComparator, label, validateVowpalWabbitString: false); } } } } public void Validate(string[] lines, JsonReader jsonReader, IVowpalWabbitLabelComparator labelComparator = null, ILabel label = null, int? index = null, VowpalWabbitJsonExtension extension = null) { VowpalWabbitExample[] strExamples = new VowpalWabbitExample[lines.Count()]; try { for (int i = 0; i < lines.Length; i++) strExamples[i] = this.vw.ParseLine(lines[i]); using (var jsonSerializer = new VowpalWabbitJsonSerializer(this.vw)) { if (extension != null) { jsonSerializer.RegisterExtension(extension); // extension are not supported with native JSON parsing } using (var jsonExample = (VowpalWabbitMultiLineExampleCollection)jsonSerializer.ParseAndCreate(jsonReader, label, index)) { var jsonExamples = new List(); if (jsonExample.SharedExample != null) jsonExamples.Add(jsonExample.SharedExample); jsonExamples.AddRange(jsonExample.Examples); Assert.AreEqual(strExamples.Length, jsonExamples.Count); for (int i = 0; i < strExamples.Length; i++) { using (var strJsonExample = this.vw.ParseLine(jsonExamples[i].VowpalWabbitString)) { var diff = strExamples[i].Diff(this.vw, jsonExamples[i], labelComparator); Assert.IsNull(diff, diff + " generated string: '" + jsonExamples[i].VowpalWabbitString + "'"); diff = strExamples[i].Diff(this.vw, strJsonExample, labelComparator); Assert.IsNull(diff, diff); } } } } } finally { foreach (var ex in strExamples) if (ex != null) ex.Dispose(); } } public void Validate(string[] lines, List examples, IVowpalWabbitLabelComparator labelComparator = null) { VowpalWabbitExample[] strExamples = new VowpalWabbitExample[lines.Count()]; try { for (int i = 0; i < lines.Length; i++) strExamples[i] = this.vw.ParseLine(lines[i]); for (int i = 0; i < strExamples.Length; i++) { var diff = strExamples[i].Diff(this.vw, examples[i], labelComparator); Assert.IsNull(diff, diff + " generated string: '" + strExamples[i].VowpalWabbitString + "'"); } } finally { foreach (var ex in strExamples) if (ex != null) ex.Dispose(); } } public void Validate(string[] lines, string json, IVowpalWabbitLabelComparator labelComparator = null, ILabel label = null, int? index = null, VowpalWabbitJsonExtension extension = null, bool enableNativeJsonValidation = true) { VowpalWabbitExample[] strExamples = new VowpalWabbitExample[lines.Count()]; try { for (int i = 0; i < lines.Length; i++) strExamples[i] = this.vw.ParseLine(lines[i]); using (var jsonSerializer = new VowpalWabbitJsonSerializer(this.vw)) { if (extension != null) { jsonSerializer.RegisterExtension(extension); // extension are not supported with native JSON parsing enableNativeJsonValidation = false; } List jsonNativeExamples = null; try { if (enableNativeJsonValidation) { jsonNativeExamples = this.vw.ParseJson(json); Assert.IsNotNull(jsonNativeExamples); } using (var jsonExample = (VowpalWabbitMultiLineExampleCollection)jsonSerializer.ParseAndCreate(json, label, index)) { var jsonExamples = new List(); if (jsonExample.SharedExample != null) jsonExamples.Add(jsonExample.SharedExample); jsonExamples.AddRange(jsonExample.Examples); Assert.AreEqual(strExamples.Length, jsonExamples.Count); if (enableNativeJsonValidation) Assert.AreEqual(strExamples.Length, jsonNativeExamples.Count); for (int i = 0; i < strExamples.Length; i++) { using (var strJsonExample = this.vw.ParseLine(jsonExamples[i].VowpalWabbitString)) { var diff = strExamples[i].Diff(this.vw, jsonExamples[i], labelComparator); Assert.IsNull(diff, diff + " generated string: '" + jsonExamples[i].VowpalWabbitString + "'"); diff = strExamples[i].Diff(this.vw, strJsonExample, labelComparator); Assert.IsNull(diff, diff); if (enableNativeJsonValidation) { diff = strExamples[i].Diff(this.vw, jsonNativeExamples[i], labelComparator); Assert.IsNull(diff, diff); } } } } } finally { if (jsonNativeExamples != null) { foreach (var ex in jsonNativeExamples) ex.Dispose(); } } } } finally { foreach (var ex in strExamples) if (ex != null) ex.Dispose(); } } public void Dispose() { this.Dispose(true); GC.SuppressFinalize(this); } private void Dispose(bool disposing) { if (disposing) { if (this.vw != null) { this.vw.Dispose(); this.vw = null; } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/VowpalWabbitExampleValidator.cs000066400000000000000000000217701323713302100257050ustar00rootroot00000000000000using Microsoft.VisualStudio.TestTools.UnitTesting; using Newtonsoft.Json; using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Text.RegularExpressions; using System.Threading.Tasks; using VW; using VW.Labels; using VW.Serializer; namespace cs_unittest { internal static class VowpalWabbitExampleValidator { internal static void Validate(string line, VowpalWabbitExample ex, IVowpalWabbitLabelComparator comparator, string args = null) { using (var vw = new VowpalWabbit(args)) using (var strExample = vw.ParseLine(line)) { var diff = strExample.Diff(vw, ex, comparator); Assert.IsNull(diff, diff + " generated string: '" + ex.VowpalWabbitString + "'"); } } } internal sealed class VowpalWabbitExampleValidator : IDisposable { private VowpalWabbit vw; private VowpalWabbit vwNative; private VowpalWabbit vwJson; private VowpalWabbitSingleExampleSerializerCompiler compiler; private Action serializer; private Action serializerNative; private IVowpalWabbitSerializer factorySerializer; private static string FixArgs(string args) { // remove model writing args = Regex.Replace(args, @"-f\s+[^ -]+", " "); // remove cache file args = Regex.Replace(args, @"-c\s+([^ -]+)?", " "); return args; } internal VowpalWabbitExampleValidator(string args) : this(new VowpalWabbitSettings(FixArgs(args))) { } internal VowpalWabbitExampleValidator(VowpalWabbitSettings settings) { var stringSettings = (VowpalWabbitSettings)settings.Clone(); stringSettings.EnableStringExampleGeneration = true; this.vw = new VowpalWabbit(stringSettings); var jsonSettings = (VowpalWabbitSettings)settings.Clone(); jsonSettings.Arguments += " --json"; this.vwJson = new VowpalWabbit(jsonSettings); this.compiler = this.vw.Serializer as VowpalWabbitSingleExampleSerializerCompiler; if (this.compiler != null) this.serializer = this.compiler.Func(this.vw.Native); this.vwNative = new VowpalWabbit(settings); this.compiler = this.vwNative.Serializer as VowpalWabbitSingleExampleSerializerCompiler; if (compiler != null) this.serializerNative = this.compiler.Func(this.vwNative.Native); this.factorySerializer = VowpalWabbitSerializerFactory.CreateSerializer(stringSettings).Create(this.vw.Native); } public void Validate(string line, TExample example, ILabel label = null) { IVowpalWabbitLabelComparator comparator; if (label == null || label == SharedLabel.Instance) { comparator = null; } else if (label is SimpleLabel) { comparator = VowpalWabbitLabelComparator.Simple; } else if (label is ContextualBanditLabel) { comparator = VowpalWabbitLabelComparator.ContextualBandit; } else { throw new ArgumentException("Label type not supported: " + label.GetType()); } using (var context = new VowpalWabbitMarshalContext(this.vw.Native)) using (var contextNative = new VowpalWabbitMarshalContext(this.vwNative.Native)) { // validate string serializer this.serializer(context, example, label); this.serializerNative(contextNative, example, label); // natively parsed string example compared against: // (1) natively build example // (2) string serialized & natively parsed string example using (var strExample = this.vw.Native.ParseLine(line)) using (var strConvertedExample = this.vw.Native.ParseLine(context.ToString())) using (var nativeExample = contextNative.ExampleBuilder.CreateExample()) using (var nativeExampleWithString = this.factorySerializer.Serialize(example, label)) { var diff = strExample.Diff(this.vw.Native, strConvertedExample, comparator); Assert.IsNull(diff, diff + " generated string: '" + context.ToString() + "'"); diff = strExample.Diff(this.vw.Native, nativeExample, comparator); Assert.IsNull(diff, diff); if (!strExample.IsNewLine) { Assert.IsFalse(string.IsNullOrEmpty(nativeExampleWithString.VowpalWabbitString)); Assert.IsFalse(string.IsNullOrEmpty(this.factorySerializer.SerializeToString(example, label))); } if (this.vw.Native.Settings.TypeInspector == JsonTypeInspector.Default) { var jsonStr = JsonConvert.SerializeObject(example); using (var jsonSerializer = new VowpalWabbitJsonSerializer(this.vw.Native)) { using (var jsonExample = jsonSerializer.ParseAndCreate(jsonStr, label)) { var ex = ((VowpalWabbitSingleLineExampleCollection)jsonExample).Example; diff = strExample.Diff(this.vw.Native, ex, comparator); Assert.IsNull(diff, $"{diff}\n json: '{jsonStr}'"); } } List exampleList = null; try { exampleList = this.vwJson.ParseJson(jsonStr); Assert.AreEqual(1, exampleList.Count); diff = strExample.Diff(this.vw.Native, exampleList[0], comparator); Assert.IsNull(diff, $"{diff}\n json: '{jsonStr}'"); } finally { if (exampleList != null) { foreach (var ex in exampleList) ex.Dispose(); } } } } } } public void Validate(IEnumerable lines, TExample example, IVowpalWabbitLabelComparator labelComparator = null, ILabel label = null) { // natively parsed string example compared against: // (1) natively build example // (2) string serialized & natively parsed string example var strExamples = lines.Select(l => this.vw.Native.ParseLine(l)).ToArray(); using (var nativeExampleWithString = (VowpalWabbitMultiLineExampleCollection)this.factorySerializer.Serialize(example, label)) { var examplesToCompare = new List(); if (nativeExampleWithString.SharedExample != null) examplesToCompare.Add(nativeExampleWithString.SharedExample); examplesToCompare.AddRange(nativeExampleWithString.Examples); examplesToCompare = examplesToCompare.Where(e => !e.IsNewLine).ToList(); Assert.AreEqual(strExamples.Length, examplesToCompare.Count); for (int i = 0; i < strExamples.Length; i++) { var diff = strExamples[i].Diff(this.vw.Native, examplesToCompare[i], labelComparator); Assert.IsNull(diff, diff + " generated string: '" + examplesToCompare[i].VowpalWabbitString + "'"); } } foreach (var ex in strExamples) ex.Dispose(); } public void Dispose() { this.Dispose(true); GC.SuppressFinalize(this); } private void Dispose(bool disposing) { if (disposing) { if (this.vw != null) { this.vw.Dispose(); this.vw = null; } if (this.vwNative != null) { this.vwNative.Dispose(); this.vwNative = null; } if (this.vwJson != null) { this.vwJson.Dispose(); this.vwJson = null; } if (this.factorySerializer != null) { this.factorySerializer.Dispose(); this.factorySerializer = null; } } } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/VowpalWabbitListenerToEvents.cs000066400000000000000000000003061323713302100257110ustar00rootroot00000000000000using System; using VW.Labels; namespace cs_unittest { public class VowpalWabbitListenerToEvents : VowpalWabbitBaseListener { public Action Created; } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/VowpalWabbitStdErrPerformanceStatistics.cs000066400000000000000000000011231323713302100300720ustar00rootroot00000000000000using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; namespace cs_unittest { class VowpalWabbitStdErrPerformanceStatistics { public double? AverageLoss { get; set; } public double? BestConstant { get; set; } public double? BestConstantLoss { get; set; } public ulong? NumberOfExamplesPerPass { get; set; } public ulong? TotalNumberOfFeatures { get; set; } public double? WeightedExampleSum { get; set; } public double? WeightedLabelSum { get; set; } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/azure.provision.ps1000066400000000000000000000004301323713302100233630ustar00rootroot00000000000000Login-AzureRmAccount Select-AzureRmSubscription -SubscriptionId "FILL ME IN" $rg = 'FILL ME IN' New-AzureRmResourceGroup -Name $rg -Location "East US" # -DeploymentDebugLogLevel All New-AzureRmResourceGroupDeployment -ResourceGroupName $rg -TemplateFile .\azuredeploy.json vowpal-wabbit-8.5.0.dfsg1/cs/unittest/azuredeploy.json000066400000000000000000000110201323713302100230140ustar00rootroot00000000000000{ "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "variables": { "storageName": "[concat(resourceGroup().name, 'storage')]", "serviceBusName": "[concat(resourceGroup().name, 'sb')]", "eventHubInputName": "[concat(resourceGroup().name, 'input')]", "eventHubEvalName": "[concat(resourceGroup().name, 'eval')]" }, "resources": [ { "type": "Microsoft.Storage/storageAccounts", "name": "[variables('storageName')]", "apiVersion": "2015-06-15", "location": "[resourceGroup().location]", "properties": { "accountType": "Standard_LRS" } }, { "apiVersion": "2015-08-01", "name": "[concat(variables('serviceBusName'))]", "type": "Microsoft.ServiceBus/namespaces", "location": "[resourceGroup().location]", "kind": "Messaging", "sku": { "name": "Standard", "tier": "Standard", "capacity": 1 }, "resources": [ { "apiVersion": "2015-08-01", "name": "[concat(variables('serviceBusName'), '/', 'OwnerKey')]", "type": "Microsoft.ServiceBus/namespaces/authorizationRules", "location": "[resourceGroup().location]", "dependsOn": [ "[concat('Microsoft.ServiceBus/namespaces/', variables('serviceBusName'))]" ], "properties": { "KeyName": "OwnerKey", "ClaimType": "SharedAccessKey", "ClaimValue": "None", "Rights": [ "listen", "send", "manage" ], "Revision": -1 } } ] }, { "apiVersion": "2015-08-01", "name": "[concat(variables('serviceBusName'), '/', variables('eventHubInputName'))]", "type": "Microsoft.ServiceBus/namespaces/eventHubs", "location": "[resourceGroup().location]", "properties": { "path": "[variables('eventHubInputName')]", "MessageRetentionInDays": 7, "PartitionCount": 2 }, "dependsOn": [ "[concat('Microsoft.ServiceBus/namespaces/', variables('serviceBusName'))]" ], "resources": [ { "apiVersion": "2015-08-01", "name": "Manage", "type": "AuthorizationRules", "dependsOn": [ "[concat(resourceId('Microsoft.ServiceBus/namespaces', variables('serviceBusName')), '/eventHubs/', variables('eventHubInputName'))]" ], "properties": { "keyName": "Manage", "claimType": "SharedAccessKey", "claimValue": "None", "rights": [ "Send", "Listen", "Manage" ] } } ] }, { "apiVersion": "2015-08-01", "name": "[concat(variables('serviceBusName'), '/', variables('eventHubEvalName'))]", "type": "Microsoft.ServiceBus/namespaces/eventHubs", "location": "[resourceGroup().location]", "properties": { "path": "[variables('eventHubEvalName')]", "MessageRetentionInDays": 7, "PartitionCount": 2 }, "dependsOn": [ "[concat('Microsoft.ServiceBus/namespaces/', variables('serviceBusName'))]" ], "resources": [ { "apiVersion": "2015-08-01", "name": "Manage", "type": "AuthorizationRules", "dependsOn": [ "[concat(resourceId('Microsoft.ServiceBus/namespaces', variables('serviceBusName')), '/eventHubs/', variables('eventHubEvalName'))]" ], "properties": { "keyName": "Manage", "claimType": "SharedAccessKey", "claimValue": "None", "rights": [ "Send", "Listen", "Manage" ] } } ] } ], "outputs": { "storageConnectionString": { "type": "string", "value": "[concat('DefaultEndpointsProtocol=https;AccountName=',variables('storageName'),';AccountKey=',listkeys(resourceId('Microsoft.Storage/storageAccounts', variables('storageName')), '2015-06-15').key1)]" }, "eventHubInputConnectionString": { "type": "string", "value": "[listkeys(resourceId('Microsoft.ServiceBus/namespaces/eventhubs/authorizationRules', variables('serviceBusName'), variables('eventHubInputName'), 'Manage'), '2015-08-01').primaryConnectionString]" }, "eventHubEvalConnectionString": { "type": "string", "value": "[listkeys(resourceId('Microsoft.ServiceBus/namespaces/eventhubs/authorizationRules', variables('serviceBusName'), variables('eventHubEvalName'), 'Manage'), '2015-08-01').primaryConnectionString]" } } } vowpal-wabbit-8.5.0.dfsg1/cs/unittest/cs_unittest.csproj000066400000000000000000000445771323713302100233730ustar00rootroot00000000000000 Debug AnyCPU {007B7DE0-1D9B-498D-ACFC-E9D33058F22E} Library Properties cs_unittest cs_unittest v4.6 512 {3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC} 10.0 $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion) $(ProgramFiles)\Common Files\microsoft shared\VSTT\$(VisualStudioVersion)\UITestExtensionPackages False UnitTest $(SolutionDir)\ true $(ProjectDir)..\..\vowpalwabbit\x64\Debug\ DEBUG;TRACE full x64 prompt MinimumRecommendedRules.ruleset $(ProjectDir)..\..\vowpalwabbit\x64\Release\ TRACE true pdbonly x64 prompt MinimumRecommendedRules.ruleset true ..\vw_key.snk true $(SolutionDir)\x64\DebugLeakCheck\ DEBUG;TRACE full x64 prompt MinimumRecommendedRules.ruleset $(SolutionDir)\packages\Antlr4.Runtime.4.6.4\lib\net45\Antlr4.Runtime.dll $(SolutionDir)\packages\Microsoft.ApplicationInsights.PerfCounterCollector.2.2.0\lib\net45\Microsoft.AI.PerfCounterCollector.dll True $(SolutionDir)\packages\Microsoft.ApplicationInsights.2.2.0\lib\net46\Microsoft.ApplicationInsights.dll True $(SolutionDir)\packages\Microsoft.Azure.KeyVault.Core.2.0.4\lib\net45\Microsoft.Azure.KeyVault.Core.dll True $(SolutionDir)\packages\Microsoft.Data.Edm.5.8.1\lib\net40\Microsoft.Data.Edm.dll True $(SolutionDir)\packages\Microsoft.Data.OData.5.8.1\lib\net40\Microsoft.Data.OData.dll True $(SolutionDir)\packages\Microsoft.Data.Services.Client.5.8.1\lib\net40\Microsoft.Data.Services.Client.dll True $(SolutionDir)\packages\WindowsAzure.ServiceBus.3.4.3\lib\net45-full\Microsoft.ServiceBus.dll True $(SolutionDir)\packages\Microsoft.Azure.ServiceBus.EventProcessorHost.2.2.8\lib\net45-full\Microsoft.ServiceBus.Messaging.EventProcessorHost.dll True $(SolutionDir)\packages\WindowsAzure.Storage.7.2.1\lib\net40\Microsoft.WindowsAzure.Storage.dll True $(SolutionDir)\packages\morelinq.1.4.0\lib\net35\MoreLinq.dll True $(SolutionDir)\packages\Newtonsoft.Json.9.0.1\lib\net45\Newtonsoft.Json.dll True $(SolutionDir)\packages\System.Reactive.Core.3.1.1\lib\net46\System.Reactive.Core.dll True $(SolutionDir)\packages\System.Reactive.Interfaces.3.1.1\lib\net45\System.Reactive.Interfaces.dll True $(SolutionDir)\packages\System.Reactive.Linq.3.1.1\lib\net46\System.Reactive.Linq.dll True $(SolutionDir)\packages\System.Reactive.PlatformServices.3.1.1\lib\net46\System.Reactive.PlatformServices.dll True $(SolutionDir)\packages\System.Reactive.Windows.Threading.3.1.1\lib\net45\System.Reactive.Windows.Threading.dll True $(SolutionDir)\packages\System.Spatial.5.8.1\lib\net40\System.Spatial.dll True $(SolutionDir)\packages\Microsoft.Tpl.Dataflow.4.5.24\lib\portable-net45+win8+wpa81\System.Threading.Tasks.Dataflow.dll True %(RecursiveDir)%(FileName)%(Extension) PreserveNewest True True RunTests.tt True True TestMarshalNumeric.tt {43e32c1d-21d6-4be3-8fa5-d5ba379cba53} azure {e4e962ae-7056-4eb0-a8c5-8dc824a4b068} cs {01a85382-c3e9-480a-86bf-fafe4ad107a7} cs_console {9e27fa94-ab34-4736-8427-fb7a2ba90d52} cs_json {08636f79-5577-4af2-8eed-ec8a5bc14ac4} cs_parallel {6a9cbeab-427f-4d8d-9559-b76b42b0895c} cs_testcommon {85e55ae0-3784-4968-9271-c81af560e1c1} vw_clr {e621e022-c1f8-433f-905a-ab9a3de072b7} vw_common PreserveNewest PreserveNewest Designer MSBuild:Compile True False TextTemplatingFileGenerator RunTests.cs TextTemplatingFileGenerator TestMarshalNumeric.cs False False False False This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. vowpal-wabbit-8.5.0.dfsg1/cs/unittest/json/000077500000000000000000000000001323713302100205355ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/cs/unittest/json/test_array.json000066400000000000000000000000501323713302100236000ustar00rootroot00000000000000[{"_label":1,"f":1}, {"_label":0,"f":2}]vowpal-wabbit-8.5.0.dfsg1/cs/unittest/json/test_newline.json000066400000000000000000000000471323713302100241310ustar00rootroot00000000000000 {"_label":1,"f":1} {"_label":0,"f":2} vowpal-wabbit-8.5.0.dfsg1/cs/unittest/packages.config000066400000000000000000000040221323713302100225270ustar00rootroot00000000000000 vowpal-wabbit-8.5.0.dfsg1/cs/version.props000066400000000000000000000004071323713302100204600ustar00rootroot00000000000000 8.4.0.1 vowpal-wabbit-8.5.0.dfsg1/cs/vw_key.snk000066400000000000000000000011241323713302100177240ustar00rootroot00000000000000$RSA2QZR+8x':{iIRShA+u"L2JƃIf^<Ȃ Ҿdƈd j忿E HVc)TYG%̢/2%x5rVI<D> u*>T3O&VF]u7c6Ոl\0)$stw_E&FMjfI?ͦfciE},ܕ؃o*pjws/kkFW;XD܄;JF]H9zrkflIg ;(WAˍ't.w$ qWF۔ʓT瑑hEڸ&|u2-Z`BBoW5G{<szȈ"ޖg#AFA>2o_pj%5&B/^縰$.gnn5>.Jԛl8- J2#(M6X:qǎK RQK,GWƹa|5H' s8|1 .wr߄.Üd1vowpal-wabbit-8.5.0.dfsg1/demo/000077500000000000000000000000001323713302100162245ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/demo/advertising/000077500000000000000000000000001323713302100205435ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/demo/advertising/naive_baseline.py000077500000000000000000000054431323713302100240720ustar00rootroot00000000000000#!/usr/bin/python import sys from subprocess import call, Popen, PIPE from math import log, exp from os import devnull # The learning algorithm is vowpal wabbit, available at https://github.com/JohnLangford/vowpal_wabbit/wiki vw_train_cmd = '../../vowpalwabbit/vw -c -f model --bfgs --passes 30 -b 22 --loss_function logistic --l2 14 --termination 0.00001 --holdout_off' vw_test_cmd = '../../vowpalwabbit/vw -t -i model -p /dev/stdout' def get_features(line): feat = line[2:] # Bucketizing the integer features on a logarithmic scale for i in range(8): if feat[i]: v = int(feat[i]) if v>0: feat[i] = str(int(log(v+0.5)/log(1.5))) return ' '.join(['%d_%s' % (i,v) for i,v in enumerate(feat) if v]) def train_test_oneday(day): ts_beginning_test = 86400*(day-1) with open('data.txt') as f: line = f.readline() # Beginning of the training set: 3 weeks before the test period while int(line.split()[0]) < ts_beginning_test - 86400*21: line = f.readline() call('rm -f .cache', shell=True) vw = Popen(vw_train_cmd, shell=True, stdin=PIPE) print '---------- Training on days %d to %d ----------------' % (day-21, day-1) print while int(line.split()[0]) < ts_beginning_test: line = line[:-1].split('\t') label = -1 if line[1]: conv_ts = int(line[1]) if conv_ts < ts_beginning_test: label = 1 # Positive label iff conversion and the conversion occured before the test period out = '%d | %s' % (label, get_features(line)) print >>vw.stdin, out line = f.readline() vw.stdin.close() vw.wait() print print '---------- Testing on day %d ----------------' % (day-21) vw = Popen(vw_test_cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=open(devnull, 'w')) ll = 0 n = 0 # Test is one day long while int(line.split()[0]) < ts_beginning_test + 86400: line = line[:-1].split('\t') print >>vw.stdin, '| '+get_features(line) dotproduct = float(vw.stdout.readline()) # Test log likelihood if line[1]: # Positive example ll += log(1+exp(-dotproduct)) else: # Negative sample ll += log(1+exp(dotproduct)) n += 1 line = f.readline() return (ll, n) def main(): ll = 0 n = 0 # Iterating over the 7 test days for day in range(54,61): ll_day, n_day = train_test_oneday(day) ll += ll_day n += n_day print ll_day, n_day print print 'Average test log likelihood: %f' % (ll/n) if __name__ == "__main__": main() vowpal-wabbit-8.5.0.dfsg1/demo/advertising/readme.txt000066400000000000000000000013471323713302100225460ustar00rootroot00000000000000 ------ Criteo Conversion Logs ------ This dataset contains feature values and conversion feedback for millions of clicks on display ads. It served to benchmark algorithms described in the paper "Modeling Delayed Feedback in Display Advertising", KDD 2014 by O.Chapelle (http://olivier.chapelle.cc/pub/delayedConv.pdf) =================================================== Instructions - Go to http://labs.criteo.com/downloads/2014-conversion-logs-dataset/ - Download the tar file after agreeing to the terms and conditions - Untar the file - Move data.txt to this directory - Run the naive_baseline.py script ==================================================== Dataset assembled by Olivier Chapelle (o.chapelle@criteo.com) vowpal-wabbit-8.5.0.dfsg1/demo/dependencyparsing/000077500000000000000000000000001323713302100217265ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/demo/dependencyparsing/Makefile000066400000000000000000000022751323713302100233740ustar00rootroot00000000000000SHELL=/bin/bash VW=../../vowpalwabbit/vw python_exe=python2.7 train_data=wsj_train_subset test_data=wsj_test_subset tag=tags .SECONDARY: all: @cat README.md clean: rm -f *.model *.predictions *.parse *.vw *.cache *~ *.writing %.check: @test -x "$$(which $*)" || { \ echo "ERROR: you need to install $*" 1>&2; \ exit 1; \ } %.vw: @echo "preprocessing dependency parse data ..." 1>&2 @echo "Note that due to the licence issue, we only provide a subset of the corpus" 1>&2 @$(python_exe) parse_data.py $(train_data) $(train_data).vw @$(python_exe) parse_data.py $(test_data) $(test_data).vw dep.model: $(train_data).vw @$(VW) --passes 3 -d $< -k -c --search_rollin mix_per_roll --search_task dep_parser --search 12 --search_alpha 1e-5 --search_rollout oracle --holdout_off -f $@ --search_history_length 3 --search_no_caching -b 30 --root_label 8 --num_label 12 --nn 5 --ftrl dep.test.predictions: $(test_data).vw dep.model @$(VW) -d $< -t $(search_option) -i dep.model -p $@ dep.test.parse: dep.test.predictions @$(python_exe) parse_test_result.py $(test_data) $< $(tag) > $@ dep.perf: dep.test.parse @$(python_exe) evaluate.py dep.test.parse $(test_data) .PHONY: all clean vowpal-wabbit-8.5.0.dfsg1/demo/dependencyparsing/README.md000077500000000000000000000010431323713302100232060ustar00rootroot00000000000000Search for dependencies ------------------------------------- This demo shows the performance of a dependency parser implemented in search framework. For more details, see http://arxiv.org/abs/1503.05615 Note that due to the licence issue, we only provide a subset of English Penn Treebank. If you have the full set of the data, please change the path in Makefile accordingly. ### Instruction ### - `make dep.perf`: downloads the subset of the English Penn Treebank corpus, trains a dependency parser, and computes test set statistics. vowpal-wabbit-8.5.0.dfsg1/demo/dependencyparsing/evaluate.py000077500000000000000000000061071323713302100241150ustar00rootroot00000000000000#!/usr/bin/env python # evaluation script modified from redshift parser import os import sys from collections import defaultdict def pc(num, den): return (num / float(den+1e-100)) * 100 def fmt_acc(label, n, l_corr, u_corr, total_errs): l_pc = pc(l_corr, n) u_pc = pc(u_corr, n) err_pc = pc(n - l_corr, total_errs) return '%s\t%d\t%.3f\t%.3f\t%.3f' % (label, n, l_pc, u_pc, err_pc) def gen_toks(loc): sent_strs = open(str(loc)).read().strip().split('\n\n') token = None i = 0 for sent_str in sent_strs: tokens = [Token(i, tok_str.split()) for i, tok_str in enumerate(sent_str.split('\n'))] for token in tokens: yield sent_str, token class Token(object): def __init__(self, id_, attrs): self.id = id_ # CoNLL format if len(attrs) == 10: new_attrs = [str(int(attrs[0]) - 1)] new_attrs.append(attrs[1]) new_attrs.append(attrs[3]) new_attrs.append(str(int(attrs[-4]) - 1)) new_attrs.append(attrs[-3]) attrs = new_attrs self.label = attrs[-1] if self.label.lower() == 'root': self.label = 'ROOT' try: head = int(attrs[-2]) except: try: self.label = 'P' head = int(attrs[-1]) except: print attrs raise attrs.pop() attrs.pop() self.head = head self.pos = attrs.pop() self.word = attrs.pop() self.dir = 'R' if head >= 0 and head < self.id else 'L' def mymain(test_loc, gold_loc, eval_punct=False): if not os.path.exists(test_loc): test_loc.mkdir() n_by_label = defaultdict(lambda: defaultdict(int)) u_by_label = defaultdict(lambda: defaultdict(int)) l_by_label = defaultdict(lambda: defaultdict(int)) N = 0 u_nc = 0 l_nc = 0 for (sst, t), (ss, g) in zip(gen_toks(test_loc), gen_toks(gold_loc)): if not eval_punct and g.word in ",.-;:'\"!?`{}()[]": continue prev_g = g prev_t = t u_c = g.head == t.head l_c = u_c and g.label.lower() == t.label.lower() N += 1 l_nc += l_c u_nc += u_c n_by_label[g.dir][g.label] += 1 u_by_label[g.dir][g.label] += u_c l_by_label[g.dir][g.label] += l_c n_l_err = N - l_nc for D in ['L', 'R']: n_other = 0 l_other = 0 u_other = 0 for label, n in sorted(n_by_label[D].items(), key=lambda i: i[1], reverse=True): if n == 0: continue elif n < 100: n_other += n l_other += l_by_label[D][label] u_other += u_by_label[D][label] else: l_corr = l_by_label[D][label] u_corr = u_by_label[D][label] yield 'U: %.3f' % pc(u_nc, N) yield 'L: %.3f' % pc(l_nc, N) if __name__ == '__main__': if(sys.argv < 3): print 'Usage: parsed_pred_file gold_test_conll_file' sys.exit(0) for line in mymain(sys.argv[1], sys.argv[2], eval_punct=False): print line vowpal-wabbit-8.5.0.dfsg1/demo/dependencyparsing/parse_data.py000066400000000000000000000014531323713302100244060ustar00rootroot00000000000000from sys import argv hash = {} def readtags(): for line in open('tags').readlines(): hash[line.split()[0]] = int(line.strip().split()[1]) if __name__ == '__main__': c = 1 readtags() if len(argv) != 3: print 'parseDepData.py input output' data = open(argv[1]).readlines() writer = open(argv[2],'w') for line in data: if line == '\n': writer.write('\n') continue splits = line.strip().lower().split() strw = "|w %s"%splits[1].replace(":","COL"); strp = "|p %s"%splits[4].replace(":","COL"); tag = splits[8] if tag not in hash: hash[tag] = c c+=1 #writer.write('%s 1.0 %s:%s%s %s\n'%((int(splits[7])+1) + (hash[tag]<<8), int(splits[7]),tag,strw, strp)) writer.write('%s %s %s:%s%s %s\n' % (int(splits[7]), hash[tag], int(splits[7]), tag, strw, strp)) writer.close() vowpal-wabbit-8.5.0.dfsg1/demo/dependencyparsing/parse_test_result.py000066400000000000000000000013661323713302100260550ustar00rootroot00000000000000from sys import argv from sys import exit dict = {} if len(argv) <4: print "Usage: test_conll_file annotation_file_from_vw tag_id_mapping" exit(1) for data in open(argv[3]).readlines(): dict[data.strip().split()[1]] = data.strip().split()[0] annotation = open(argv[2]).readlines() #for item in list(annotation): # if item == ' w\n': # annotation.remove(item) for idx, line in enumerate(open(argv[1]).readlines()): item = line.split() # conll07 if len(item) ==10: item[-4] = annotation[idx].strip().split(":")[0] item[-3] = dict[annotation[idx].strip().split(":")[1]] # wsj corpus elif len(item) >0: # print idx item[-2] = annotation[idx].strip().split(":")[0] item[-1] = dict[annotation[idx].strip().split(":")[1]] print "\t".join(item) vowpal-wabbit-8.5.0.dfsg1/demo/dependencyparsing/tags000066400000000000000000000001161323713302100226050ustar00rootroot00000000000000dep 6 vc 9 obj 7 sub 5 vmod 1 prd 10 pmod 3 nmod 2 amod 11 p 4 sbar 12 root 8 vowpal-wabbit-8.5.0.dfsg1/demo/dependencyparsing/wsj_test_subset000066400000000000000000001243031323713302100251030ustar00rootroot000000000000001 No _ RB RB ADV _ 4 VMOD 2 , _ , , . _ 4 P 3 it _ PR PRP PRON _ 4 SUB 4 was _ VB VBD VERB _ 0 ROOT 5 n't _ RB RB ADV _ 4 VMOD 6 Black _ NN NNP NOUN _ 7 NMOD 7 Monday _ NN NNP NOUN _ 4 PRD 8 . _ . . . _ 4 P 1 But _ CC CC CONJ _ 33 VMOD 2 while _ IN IN ADP _ 33 VMOD 3 the _ DT DT DET _ 7 NMOD 4 New _ NN NNP NOUN _ 7 NMOD 5 York _ NN NNP NOUN _ 7 NMOD 6 Stock _ NN NNP NOUN _ 7 NMOD 7 Exchange _ NN NNP NOUN _ 8 SUB 8 did _ VB VBD VERB _ 2 SBAR 9 n't _ RB RB ADV _ 8 VMOD 10 fall _ VB VB VERB _ 8 VC 11 apart _ RB RB ADV _ 10 VMOD 12 Friday _ NN NNP NOUN _ 10 VMOD 13 as _ IN IN ADP _ 10 VMOD 14 the _ DT DT DET _ 18 NMOD 15 Dow _ NN NNP NOUN _ 18 NMOD 16 Jones _ NN NNP NOUN _ 18 NMOD 17 Industrial _ NN NNP NOUN _ 18 NMOD 18 Average _ NN NNP NOUN _ 19 SUB 19 plunged _ VB VBD VERB _ 13 SBAR 20 190.58 _ CD CD NUM _ 21 NMOD 21 points _ NN NNS NOUN _ 19 VMOD 22 -- _ : : . _ 23 P 23 most _ JJ JJS ADJ _ 21 NMOD 24 of _ IN IN ADP _ 23 NMOD 25 it _ PR PRP PRON _ 24 PMOD 26 in _ IN IN ADP _ 23 NMOD 27 the _ DT DT DET _ 29 NMOD 28 final _ JJ JJ ADJ _ 29 NMOD 29 hour _ NN NN NOUN _ 26 PMOD 30 -- _ : : . _ 23 P 31 it _ PR PRP PRON _ 33 SUB 32 barely _ RB RB ADV _ 33 VMOD 33 managed _ VB VBD VERB _ 0 ROOT 34 to _ TO TO PRT _ 35 VMOD 35 stay _ VB VB VERB _ 33 VMOD 36 this _ DT DT DET _ 37 NMOD 37 side _ NN NN NOUN _ 35 VMOD 38 of _ IN IN ADP _ 37 NMOD 39 chaos _ NN NN NOUN _ 38 PMOD 40 . _ . . . _ 33 P 1 Some _ DT DT DET _ 4 NMOD 2 `` _ `` `` . _ 4 P 3 circuit _ NN NN NOUN _ 4 NMOD 4 breakers _ NN NNS NOUN _ 12 SUB 5 '' _ '' '' . _ 4 P 6 installed _ VB VBN VERB _ 4 NMOD 7 after _ IN IN ADP _ 6 VMOD 8 the _ DT DT DET _ 11 NMOD 9 October _ NN NNP NOUN _ 11 NMOD 10 1987 _ CD CD NUM _ 11 NMOD 11 crash _ NN NN NOUN _ 7 PMOD 12 failed _ VB VBD VERB _ 0 ROOT 13 their _ PR PRP$ PRON _ 15 NMOD 14 first _ JJ JJ ADJ _ 15 NMOD 15 test _ NN NN NOUN _ 12 OBJ 16 , _ , , . _ 18 P 17 traders _ NN NNS NOUN _ 18 SUB 18 say _ VB VBP VERB _ 12 VMOD 19 , _ , , . _ 18 P 20 unable _ JJ JJ ADJ _ 12 VMOD 21 to _ TO TO PRT _ 22 VMOD 22 cool _ VB VB VERB _ 20 AMOD 23 the _ DT DT DET _ 25 NMOD 24 selling _ NN NN NOUN _ 25 NMOD 25 panic _ NN NN NOUN _ 22 OBJ 26 in _ IN IN ADP _ 25 NMOD 27 both _ DT DT DET _ 30 NMOD 28 stocks _ NN NNS NOUN _ 30 NMOD 29 and _ CC CC CONJ _ 30 NMOD 30 futures _ NN NNS NOUN _ 26 PMOD 31 . _ . . . _ 12 P 1 The _ DT DT DET _ 5 NMOD 2 49 _ CD CD NUM _ 5 NMOD 3 stock _ NN NN NOUN _ 5 NMOD 4 specialist _ NN NN NOUN _ 5 NMOD 5 firms _ NN NNS NOUN _ 15 NMOD 6 on _ IN IN ADP _ 5 NMOD 7 the _ DT DT DET _ 10 NMOD 8 Big _ NN NNP NOUN _ 10 NMOD 9 Board _ NN NNP NOUN _ 10 NMOD 10 floor _ NN NN NOUN _ 6 PMOD 11 -- _ : : . _ 15 P 12 the _ DT DT DET _ 15 NMOD 13 buyers _ NN NNS NOUN _ 15 NMOD 14 and _ CC CC CONJ _ 15 NMOD 15 sellers _ NN NNS NOUN _ 29 SUB 16 of _ IN IN ADP _ 15 NMOD 17 last _ JJ JJ ADJ _ 18 NMOD 18 resort _ NN NN NOUN _ 16 PMOD 19 who _ WP WP PRON _ 15 NMOD 20 were _ VB VBD VERB _ 19 SBAR 21 criticized _ VB VBN VERB _ 20 VC 22 after _ IN IN ADP _ 21 VMOD 23 the _ DT DT DET _ 25 NMOD 24 1987 _ CD CD NUM _ 25 NMOD 25 crash _ NN NN NOUN _ 22 PMOD 26 -- _ : : . _ 15 P 27 once _ RB RB ADV _ 29 VMOD 28 again _ RB RB ADV _ 27 AMOD 29 could _ MD MD VERB _ 0 ROOT 30 n't _ RB RB ADV _ 29 VMOD 31 handle _ VB VB VERB _ 29 VC 32 the _ DT DT DET _ 34 NMOD 33 selling _ NN NN NOUN _ 34 NMOD 34 pressure _ NN NN NOUN _ 31 OBJ 35 . _ . . . _ 29 P 1 Big _ JJ JJ ADJ _ 3 NMOD 2 investment _ NN NN NOUN _ 3 NMOD 3 banks _ NN NNS NOUN _ 4 SUB 4 refused _ VB VBD VERB _ 25 VMOD 5 to _ TO TO PRT _ 6 VMOD 6 step _ VB VB VERB _ 4 VMOD 7 up _ IN IN ADP _ 6 VMOD 8 to _ TO TO PRT _ 7 AMOD 9 the _ DT DT DET _ 10 NMOD 10 plate _ NN NN NOUN _ 8 PMOD 11 to _ TO TO PRT _ 12 VMOD 12 support _ VB VB VERB _ 6 VMOD 13 the _ DT DT DET _ 16 NMOD 14 beleaguered _ JJ JJ ADJ _ 16 NMOD 15 floor _ NN NN NOUN _ 16 NMOD 16 traders _ NN NNS NOUN _ 12 OBJ 17 by _ IN IN ADP _ 12 VMOD 18 buying _ VB VBG VERB _ 17 PMOD 19 big _ JJ JJ ADJ _ 20 NMOD 20 blocks _ NN NNS NOUN _ 18 OBJ 21 of _ IN IN ADP _ 20 NMOD 22 stock _ NN NN NOUN _ 21 PMOD 23 , _ , , . _ 25 P 24 traders _ NN NNS NOUN _ 25 SUB 25 say _ VB VBP VERB _ 0 ROOT 26 . _ . . . _ 25 P 1 Heavy _ JJ JJ ADJ _ 2 NMOD 2 selling _ NN NN NOUN _ 14 SUB 3 of _ IN IN ADP _ 2 NMOD 4 Standard _ NN NNP NOUN _ 7 NMOD 5 & _ CC CC CONJ _ 7 NMOD 6 Poor _ NN NNP NOUN _ 7 NMOD 7 's _ PO POS PRT _ 10 NMOD 8 500-stock _ JJ JJ ADJ _ 10 NMOD 9 index _ NN NN NOUN _ 10 NMOD 10 futures _ NN NNS NOUN _ 3 PMOD 11 in _ IN IN ADP _ 2 NMOD 12 Chicago _ NN NNP NOUN _ 11 PMOD 13 relentlessly _ RB RB ADV _ 14 VMOD 14 beat _ VB VBD VERB _ 0 ROOT 15 stocks _ NN NNS NOUN _ 14 OBJ 16 downward _ RB RB ADV _ 14 VMOD 17 . _ . . . _ 14 P 1 Seven _ CD CD NUM _ 4 NMOD 2 Big _ NN NNP NOUN _ 4 NMOD 3 Board _ NN NNP NOUN _ 4 NMOD 4 stocks _ NN NNS NOUN _ 23 NMOD 5 -- _ : : . _ 23 P 6 UAL _ NN NNP NOUN _ 23 NMOD 7 , _ , , . _ 23 P 8 AMR _ NN NNP NOUN _ 23 NMOD 9 , _ , , . _ 23 P 10 BankAmerica _ NN NNP NOUN _ 23 NMOD 11 , _ , , . _ 23 P 12 Walt _ NN NNP NOUN _ 13 NMOD 13 Disney _ NN NNP NOUN _ 23 NMOD 14 , _ , , . _ 23 P 15 Capital _ NN NNP NOUN _ 16 NMOD 16 Cities\/ABC _ NN NNP NOUN _ 23 NMOD 17 , _ , , . _ 23 P 18 Philip _ NN NNP NOUN _ 19 NMOD 19 Morris _ NN NNP NOUN _ 23 NMOD 20 and _ CC CC CONJ _ 23 NMOD 21 Pacific _ NN NNP NOUN _ 23 NMOD 22 Telesis _ NN NNP NOUN _ 23 NMOD 23 Group _ NN NNP NOUN _ 25 SUB 24 -- _ : : . _ 23 P 25 stopped _ VB VBD VERB _ 0 ROOT 26 trading _ VB VBG VERB _ 25 VMOD 27 and _ CC CC CONJ _ 25 VMOD 28 never _ RB RB ADV _ 29 VMOD 29 resumed _ VB VBD VERB _ 25 VMOD 30 . _ . . . _ 25 P 1 The _ DT DT DET _ 2 NMOD 2 finger-pointing _ NN NN NOUN _ 3 SUB 3 has _ VB VBZ VERB _ 0 ROOT 4 already _ RB RB ADV _ 3 VMOD 5 begun _ VB VBN VERB _ 3 VC 6 . _ . . . _ 3 P 1 `` _ `` `` . _ 5 P 2 The _ DT DT DET _ 4 NMOD 3 equity _ NN NN NOUN _ 4 NMOD 4 market _ NN NN NOUN _ 5 SUB 5 was _ VB VBD VERB _ 0 ROOT 6 illiquid _ JJ JJ ADJ _ 5 PRD 7 . _ . . . _ 5 P 1 Once _ RB RB ADV _ 7 VMOD 2 again _ RB RB ADV _ 1 AMOD 3 { _ -L -LRB- . _ 7 P 4 the _ DT DT DET _ 5 NMOD 5 specialists _ NN NNS NOUN _ 7 SUB 6 } _ -R -RRB- . _ 7 P 7 were _ VB VBD VERB _ 25 VMOD 8 not _ RB RB ADV _ 7 VMOD 9 able _ JJ JJ ADJ _ 7 PRD 10 to _ TO TO PRT _ 11 VMOD 11 handle _ VB VB VERB _ 9 AMOD 12 the _ DT DT DET _ 13 NMOD 13 imbalances _ NN NNS NOUN _ 11 OBJ 14 on _ IN IN ADP _ 13 NMOD 15 the _ DT DT DET _ 16 NMOD 16 floor _ NN NN NOUN _ 14 PMOD 17 of _ IN IN ADP _ 16 NMOD 18 the _ DT DT DET _ 22 NMOD 19 New _ NN NNP NOUN _ 22 NMOD 20 York _ NN NNP NOUN _ 22 NMOD 21 Stock _ NN NNP NOUN _ 22 NMOD 22 Exchange _ NN NNP NOUN _ 17 PMOD 23 , _ , , . _ 25 P 24 '' _ '' '' . _ 25 P 25 said _ VB VBD VERB _ 0 ROOT 26 Christopher _ NN NNP NOUN _ 27 NMOD 27 Pedersen _ NN NNP NOUN _ 31 NMOD 28 , _ , , . _ 31 P 29 senior _ JJ JJ ADJ _ 31 NMOD 30 vice _ NN NN NOUN _ 31 NMOD 31 president _ NN NN NOUN _ 25 SUB 32 at _ IN IN ADP _ 31 NMOD 33 Twenty-First _ NN NNP NOUN _ 35 NMOD 34 Securities _ NN NNP NOUN _ 35 NMOD 35 Corp _ NN NNP NOUN _ 32 PMOD 36 . _ . . . _ 25 P 1 Countered _ VB VBD VERB _ 0 ROOT 2 James _ NN NNP NOUN _ 3 NMOD 3 Maguire _ NN NNP NOUN _ 5 NMOD 4 , _ , , . _ 5 P 5 chairman _ NN NN NOUN _ 1 SUB 6 of _ IN IN ADP _ 5 NMOD 7 specialists _ NN NNS NOUN _ 10 NMOD 8 Henderson _ NN NNP NOUN _ 10 NMOD 9 Brothers _ NN NNP NOUN _ 10 NMOD 10 Inc. _ NN NNP NOUN _ 6 PMOD 11 : _ : : . _ 1 P 12 `` _ `` `` . _ 1 P 13 It _ PR PRP PRON _ 14 SUB 14 is _ VB VBZ VERB _ 1 VMOD 15 easy _ JJ JJ ADJ _ 14 PRD 16 to _ TO TO PRT _ 17 VMOD 17 say _ VB VB VERB _ 14 VMOD 18 the _ DT DT DET _ 19 NMOD 19 specialist _ NN NN NOUN _ 20 SUB 20 is _ VB VBZ VERB _ 17 VMOD 21 n't _ RB RB ADV _ 20 VMOD 22 doing _ VB VBG VERB _ 20 VC 23 his _ PR PRP$ PRON _ 24 NMOD 24 job _ NN NN NOUN _ 22 OBJ 25 . _ . . . _ 1 P 1 When _ WR WRB ADV _ 12 VMOD 2 the _ DT DT DET _ 3 NMOD 3 dollar _ NN NN NOUN _ 4 SUB 4 is _ VB VBZ VERB _ 1 SBAR 5 in _ IN IN ADP _ 4 VMOD 6 a _ DT DT DET _ 7 NMOD 7 free-fall _ NN NN NOUN _ 5 PMOD 8 , _ , , . _ 12 P 9 even _ RB RB ADV _ 11 NMOD 10 central _ JJ JJ ADJ _ 11 NMOD 11 banks _ NN NNS NOUN _ 12 SUB 12 ca _ MD MD VERB _ 0 ROOT 13 n't _ RB RB ADV _ 12 VMOD 14 stop _ VB VB VERB _ 12 VC 15 it _ PR PRP PRON _ 14 OBJ 16 . _ . . . _ 12 P 1 Speculators _ NN NNS NOUN _ 2 SUB 2 are _ VB VBP VERB _ 0 ROOT 3 calling _ VB VBG VERB _ 2 VC 4 for _ IN IN ADP _ 3 VMOD 5 a _ DT DT DET _ 6 NMOD 6 degree _ NN NN NOUN _ 4 PMOD 7 of _ IN IN ADP _ 6 NMOD 8 liquidity _ NN NN NOUN _ 7 PMOD 9 that _ WD WDT DET _ 6 NMOD 10 is _ VB VBZ VERB _ 9 SBAR 11 not _ RB RB ADV _ 10 VMOD 12 there _ RB RB ADV _ 10 VMOD 13 in _ IN IN ADP _ 10 VMOD 14 the _ DT DT DET _ 15 NMOD 15 market _ NN NN NOUN _ 13 PMOD 16 . _ . . . _ 2 P 17 '' _ '' '' . _ 2 P 1 Many _ JJ JJ ADJ _ 3 NMOD 2 money _ NN NN NOUN _ 3 NMOD 3 managers _ NN NNS NOUN _ 6 NMOD 4 and _ CC CC CONJ _ 6 NMOD 5 some _ DT DT DET _ 6 NMOD 6 traders _ NN NNS NOUN _ 7 SUB 7 had _ VB VBD VERB _ 0 ROOT 8 already _ RB RB ADV _ 7 VMOD 9 left _ VB VBN VERB _ 7 VC 10 their _ PR PRP$ PRON _ 11 NMOD 11 offices _ NN NNS NOUN _ 9 OBJ 12 early _ RB RB ADV _ 14 NMOD 13 Friday _ NN NNP NOUN _ 14 NMOD 14 afternoon _ NN NN NOUN _ 9 VMOD 15 on _ IN IN ADP _ 9 VMOD 16 a _ DT DT DET _ 19 NMOD 17 warm _ JJ JJ ADJ _ 19 NMOD 18 autumn _ NN NN NOUN _ 19 NMOD 19 day _ NN NN NOUN _ 15 PMOD 20 -- _ : : . _ 9 P 21 because _ IN IN ADP _ 9 VMOD 22 the _ DT DT DET _ 24 NMOD 23 stock _ NN NN NOUN _ 24 NMOD 24 market _ NN NN NOUN _ 25 SUB 25 was _ VB VBD VERB _ 21 SBAR 26 so _ RB RB ADV _ 27 AMOD 27 quiet _ JJ JJ ADJ _ 25 PRD 28 . _ . . . _ 7 P 1 Then _ RB RB ADV _ 15 VMOD 2 in _ IN IN ADP _ 15 VMOD 3 a _ DT DT DET _ 5 NMOD 4 lightning _ NN NN NOUN _ 5 NMOD 5 plunge _ NN NN NOUN _ 2 PMOD 6 , _ , , . _ 15 P 7 the _ DT DT DET _ 10 NMOD 8 Dow _ NN NNP NOUN _ 10 NMOD 9 Jones _ NN NNP NOUN _ 10 NMOD 10 industrials _ NN NNS NOUN _ 15 SUB 11 in _ IN IN ADP _ 15 VMOD 12 barely _ RB RB ADV _ 14 NMOD 13 an _ DT DT DET _ 12 AMOD 14 hour _ NN NN NOUN _ 11 PMOD 15 surrendered _ VB VBD VERB _ 0 ROOT 16 about _ RB RB ADV _ 18 NMOD 17 a _ DT DT DET _ 16 AMOD 18 third _ JJ JJ ADJ _ 15 OBJ 19 of _ IN IN ADP _ 18 NMOD 20 their _ PR PRP$ PRON _ 21 NMOD 21 gains _ NN NNS NOUN _ 23 NMOD 22 this _ DT DT DET _ 23 NMOD 23 year _ NN NN NOUN _ 19 PMOD 24 , _ , , . _ 15 P 25 chalking _ VB VBG VERB _ 15 VMOD 26 up _ RP RP PRT _ 25 VMOD 27 a _ DT DT DET _ 34 NMOD 28 190.58-point _ JJ JJ ADJ _ 32 AMOD 29 , _ , , . _ 32 P 30 or _ CC CC CONJ _ 32 AMOD 31 6.9 _ CD CD NUM _ 32 AMOD 32 % _ NN NN NOUN _ 34 NMOD 33 , _ , , . _ 32 P 34 loss _ NN NN NOUN _ 25 OBJ 35 on _ IN IN ADP _ 34 NMOD 36 the _ DT DT DET _ 37 NMOD 37 day _ NN NN NOUN _ 35 PMOD 38 in _ IN IN ADP _ 25 VMOD 39 gargantuan _ JJ JJ ADJ _ 41 NMOD 40 trading _ NN NN NOUN _ 41 NMOD 41 volume _ NN NN NOUN _ 38 PMOD 42 . _ . . . _ 15 P 1 Final-hour _ JJ JJ ADJ _ 2 NMOD 2 trading _ NN NN NOUN _ 3 SUB 3 accelerated _ VB VBD VERB _ 0 ROOT 4 to _ TO TO PRT _ 3 VMOD 5 108.1 _ CD CD NUM _ 6 AMOD 6 million _ CD CD NUM _ 7 NMOD 7 shares _ NN NNS NOUN _ 10 NMOD 8 , _ , , . _ 10 P 9 a _ DT DT DET _ 10 NMOD 10 record _ NN NN NOUN _ 4 PMOD 11 for _ IN IN ADP _ 10 NMOD 12 the _ DT DT DET _ 14 NMOD 13 Big _ NN NNP NOUN _ 14 NMOD 14 Board _ NN NNP NOUN _ 11 PMOD 15 . _ . . . _ 3 P 1 At _ IN IN ADP _ 11 VMOD 2 the _ DT DT DET _ 3 NMOD 3 end _ NN NN NOUN _ 1 PMOD 4 of _ IN IN ADP _ 3 NMOD 5 the _ DT DT DET _ 6 NMOD 6 day _ NN NN NOUN _ 4 PMOD 7 , _ , , . _ 11 P 8 251.2 _ CD CD NUM _ 9 AMOD 9 million _ CD CD NUM _ 10 NMOD 10 shares _ NN NNS NOUN _ 11 SUB 11 were _ VB VBD VERB _ 0 ROOT 12 traded _ VB VBN VERB _ 11 VC 13 . _ . . . _ 11 P 1 The _ DT DT DET _ 4 NMOD 2 Dow _ NN NNP NOUN _ 4 NMOD 3 Jones _ NN NNP NOUN _ 4 NMOD 4 industrials _ NN NNS NOUN _ 5 SUB 5 closed _ VB VBD VERB _ 0 ROOT 6 at _ IN IN ADP _ 5 VMOD 7 2569.26 _ CD CD NUM _ 6 PMOD 8 . _ . . . _ 5 P 1 The _ DT DT DET _ 3 NMOD 2 Dow _ NN NNP NOUN _ 3 NMOD 3 's _ PO POS PRT _ 4 NMOD 4 decline _ NN NN NOUN _ 5 SUB 5 was _ VB VBD VERB _ 0 ROOT 6 second _ JJ JJ ADJ _ 5 PRD 7 in _ IN IN ADP _ 6 AMOD 8 point _ NN NN NOUN _ 9 NMOD 9 terms _ NN NNS NOUN _ 7 PMOD 10 only _ RB RB ADV _ 11 PMOD 11 to _ TO TO PRT _ 6 AMOD 12 the _ DT DT DET _ 16 NMOD 13 508-point _ JJ JJ ADJ _ 16 NMOD 14 Black _ NN NNP NOUN _ 16 NMOD 15 Monday _ NN NNP NOUN _ 16 NMOD 16 crash _ NN NN NOUN _ 11 PMOD 17 that _ WD WDT DET _ 16 NMOD 18 occurred _ VB VBD VERB _ 17 SBAR 19 Oct. _ NN NNP NOUN _ 18 VMOD 20 19 _ CD CD NUM _ 19 NMOD 21 , _ , , . _ 19 P 22 1987 _ CD CD NUM _ 19 NMOD 23 . _ . . . _ 5 P 1 In _ IN IN ADP _ 11 VMOD 2 percentage _ NN NN NOUN _ 3 NMOD 3 terms _ NN NNS NOUN _ 1 PMOD 4 , _ , , . _ 11 P 5 however _ RB RB ADV _ 11 VMOD 6 , _ , , . _ 11 P 7 the _ DT DT DET _ 9 NMOD 8 Dow _ NN NNP NOUN _ 9 NMOD 9 's _ PO POS PRT _ 10 NMOD 10 dive _ NN NN NOUN _ 11 SUB 11 was _ VB VBD VERB _ 0 ROOT 12 the _ DT DT DET _ 13 NMOD 13 12th-worst _ JJ JJ ADJ _ 17 NMOD 14 ever _ RB RB ADV _ 13 NMOD 15 and _ CC CC CONJ _ 17 NMOD 16 the _ DT DT DET _ 17 NMOD 17 sharpest _ JJ JJS ADJ _ 11 PRD 18 since _ IN IN ADP _ 17 NMOD 19 the _ DT DT DET _ 20 NMOD 20 market _ NN NN NOUN _ 21 SUB 21 fell _ VB VBD VERB _ 18 SBAR 22 156.83 _ CD CD NUM _ 26 NMOD 23 , _ , , . _ 26 P 24 or _ CC CC CONJ _ 26 NMOD 25 8 _ CD CD NUM _ 26 NMOD 26 % _ NN NN NOUN _ 21 VMOD 27 , _ , , . _ 21 P 28 a _ DT DT DET _ 29 NMOD 29 week _ NN NN NOUN _ 30 PMOD 30 after _ IN IN ADP _ 21 VMOD 31 Black _ NN NNP NOUN _ 32 NMOD 32 Monday _ NN NNP NOUN _ 30 PMOD 33 . _ . . . _ 11 P 1 The _ DT DT DET _ 2 NMOD 2 Dow _ NN NNP NOUN _ 3 SUB 3 fell _ VB VBD VERB _ 0 ROOT 4 22.6 _ CD CD NUM _ 5 NMOD 5 % _ NN NN NOUN _ 3 VMOD 6 on _ IN IN ADP _ 3 VMOD 7 Black _ NN NNP NOUN _ 8 NMOD 8 Monday _ NN NNP NOUN _ 6 PMOD 9 . _ . . . _ 3 P 1 Shares _ NN NNP NOUN _ 11 SUB 2 of _ IN IN ADP _ 1 NMOD 3 UAL _ NN NNP NOUN _ 6 NMOD 4 , _ , , . _ 6 P 5 the _ DT DT DET _ 6 NMOD 6 parent _ NN NN NOUN _ 2 PMOD 7 of _ IN IN ADP _ 6 NMOD 8 United _ NN NNP NOUN _ 9 NMOD 9 Airlines _ NN NNP NOUN _ 7 PMOD 10 , _ , , . _ 6 P 11 were _ VB VBD VERB _ 0 ROOT 12 extremely _ RB RB ADV _ 13 AMOD 13 active _ JJ JJ ADJ _ 11 PRD 14 all _ DT DT DET _ 15 NMOD 15 day _ NN NN NOUN _ 11 VMOD 16 Friday _ NN NNP NOUN _ 11 VMOD 17 , _ , , . _ 11 P 18 reacting _ VB VBG VERB _ 11 VMOD 19 to _ TO TO PRT _ 18 VMOD 20 news _ NN NN NOUN _ 22 NMOD 21 and _ CC CC CONJ _ 22 NMOD 22 rumors _ NN NNS NOUN _ 19 PMOD 23 about _ IN IN ADP _ 22 NMOD 24 the _ DT DT DET _ 29 NMOD 25 proposed _ VB VBN VERB _ 29 NMOD 26 $ _ $ $ . _ 29 P 27 6.79 _ CD CD NUM _ 26 AMOD 28 billion _ CD CD NUM _ 26 AMOD 29 buy-out _ NN NN NOUN _ 23 PMOD 30 of _ IN IN ADP _ 29 NMOD 31 the _ DT DT DET _ 32 NMOD 32 airline _ NN NN NOUN _ 30 PMOD 33 by _ IN IN ADP _ 29 NMOD 34 an _ DT DT DET _ 36 NMOD 35 employee-management _ JJ JJ ADJ _ 36 NMOD 36 group _ NN NN NOUN _ 33 PMOD 37 . _ . . . _ 11 P 1 Wall _ NN NNP NOUN _ 3 NMOD 2 Street _ NN NNP NOUN _ 3 NMOD 3 's _ PO POS PRT _ 10 NMOD 4 takeover-stock _ JJ JJ ADJ _ 5 NMOD 5 speculators _ NN NNS NOUN _ 10 NMOD 6 , _ , , . _ 10 P 7 or _ CC CC CONJ _ 10 NMOD 8 `` _ `` `` . _ 10 P 9 risk _ NN NN NOUN _ 10 NMOD 10 arbitragers _ NN NNS NOUN _ 13 SUB 11 , _ , , . _ 10 P 12 '' _ '' '' . _ 10 P 13 had _ VB VBD VERB _ 0 ROOT 14 placed _ VB VBN VERB _ 13 VC 15 unusually _ RB RB ADV _ 16 AMOD 16 large _ JJ JJ ADJ _ 17 NMOD 17 bets _ NN NNS NOUN _ 14 OBJ 18 that _ IN IN ADP _ 17 NMOD 19 a _ DT DT DET _ 20 NMOD 20 takeover _ NN NN NOUN _ 21 SUB 21 would _ MD MD VERB _ 26 VMOD 22 succeed _ VB VB VERB _ 21 VC 23 and _ CC CC CONJ _ 26 VMOD 24 UAL _ NN NNP NOUN _ 25 NMOD 25 stock _ NN NN NOUN _ 26 SUB 26 would _ MD MD VERB _ 18 SBAR 27 rise _ VB VB VERB _ 26 VC 28 . _ . . . _ 13 P 1 At _ IN IN ADP _ 6 VMOD 2 2:43 _ CD CD NUM _ 4 NMOD 3 p.m. _ RB RB ADV _ 4 NMOD 4 EDT _ NN NNP NOUN _ 1 PMOD 5 , _ , , . _ 6 P 6 came _ VB VBD VERB _ 14 VMOD 7 the _ DT DT DET _ 9 NMOD 8 sickening _ JJ JJ ADJ _ 9 NMOD 9 news _ NN NN NOUN _ 6 SUB 10 : _ : : . _ 14 P 11 The _ DT DT DET _ 13 NMOD 12 Big _ NN NNP NOUN _ 13 NMOD 13 Board _ NN NNP NOUN _ 14 SUB 14 was _ VB VBD VERB _ 0 ROOT 15 halting _ VB VBG VERB _ 14 VC 16 trading _ VB VBG VERB _ 15 OBJ 17 in _ IN IN ADP _ 16 NMOD 18 UAL _ NN NNP NOUN _ 17 PMOD 19 , _ , , . _ 15 P 20 `` _ `` `` . _ 15 P 21 pending _ VB VBG VERB _ 15 VMOD 22 news _ NN NN NOUN _ 21 PMOD 23 . _ . . . _ 14 P 24 '' _ '' '' . _ 14 P 1 On _ IN IN ADP _ 15 VMOD 2 the _ DT DT DET _ 4 NMOD 3 exchange _ NN NN NOUN _ 4 NMOD 4 floor _ NN NN NOUN _ 1 PMOD 5 , _ , , . _ 15 P 6 `` _ `` `` . _ 15 P 7 as _ RB RB ADV _ 15 VMOD 8 soon _ RB RB ADV _ 7 AMOD 9 as _ IN IN ADP _ 7 AMOD 10 UAL _ NN NNP NOUN _ 11 SUB 11 stopped _ VB VBD VERB _ 9 SBAR 12 trading _ VB VBG VERB _ 11 VMOD 13 , _ , , . _ 15 P 14 we _ PR PRP PRON _ 15 SUB 15 braced _ VB VBD VERB _ 21 VMOD 16 for _ IN IN ADP _ 15 VMOD 17 a _ DT DT DET _ 18 NMOD 18 panic _ NN NN NOUN _ 16 PMOD 19 , _ , , . _ 21 P 20 '' _ '' '' . _ 21 P 21 said _ VB VBD VERB _ 0 ROOT 22 one _ CD CD NUM _ 25 NMOD 23 top _ JJ JJ ADJ _ 25 NMOD 24 floor _ NN NN NOUN _ 25 NMOD 25 trader _ NN NN NOUN _ 21 SUB 26 . _ . . . _ 21 P 1 Several _ JJ JJ ADJ _ 2 NMOD 2 traders _ NN NNS NOUN _ 3 SUB 3 could _ MD MD VERB _ 0 ROOT 4 be _ VB VB VERB _ 3 VC 5 seen _ VB VBN VERB _ 4 VC 6 shaking _ VB VBG VERB _ 5 VMOD 7 their _ PR PRP$ PRON _ 8 NMOD 8 heads _ NN NNS NOUN _ 6 OBJ 9 when _ WR WRB ADV _ 5 VMOD 10 the _ DT DT DET _ 11 NMOD 11 news _ NN NN NOUN _ 12 SUB 12 flashed _ VB VBD VERB _ 9 SBAR 13 . _ . . . _ 3 P 1 For _ IN IN ADP _ 6 VMOD 2 weeks _ NN NNS NOUN _ 1 PMOD 3 , _ , , . _ 6 P 4 the _ DT DT DET _ 5 NMOD 5 market _ NN NN NOUN _ 6 SUB 6 had _ VB VBD VERB _ 0 ROOT 7 been _ VB VBN VERB _ 6 VC 8 nervous _ JJ JJ ADJ _ 7 PRD 9 about _ IN IN ADP _ 8 AMOD 10 takeovers _ NN NNS NOUN _ 9 PMOD 11 , _ , , . _ 7 P 12 after _ IN IN ADP _ 7 VMOD 13 Campeau _ NN NNP NOUN _ 15 NMOD 14 Corp. _ NN NNP NOUN _ 15 NMOD 15 's _ PO POS PRT _ 17 NMOD 16 cash _ NN NN NOUN _ 17 NMOD 17 crunch _ NN NN NOUN _ 18 SUB 18 spurred _ VB VBD VERB _ 12 SBAR 19 concern _ NN NN NOUN _ 18 OBJ 20 about _ IN IN ADP _ 19 NMOD 21 the _ DT DT DET _ 22 NMOD 22 prospects _ NN NNS NOUN _ 20 PMOD 23 for _ IN IN ADP _ 22 NMOD 24 future _ JJ JJ ADJ _ 27 NMOD 25 highly _ RB RB ADV _ 26 AMOD 26 leveraged _ JJ JJ ADJ _ 27 NMOD 27 takeovers _ NN NNS NOUN _ 23 PMOD 28 . _ . . . _ 6 P 1 And _ CC CC CONJ _ 9 VMOD 2 10 _ CD CD NUM _ 3 NMOD 3 minutes _ NN NNS NOUN _ 4 PMOD 4 after _ IN IN ADP _ 9 VMOD 5 the _ DT DT DET _ 8 NMOD 6 UAL _ NN NNP NOUN _ 8 NMOD 7 trading _ NN NN NOUN _ 8 NMOD 8 halt _ NN NN NOUN _ 4 PMOD 9 came _ VB VBD VERB _ 0 ROOT 10 news _ NN NN NOUN _ 9 SUB 11 that _ IN IN ADP _ 10 NMOD 12 the _ DT DT DET _ 14 NMOD 13 UAL _ NN NNP NOUN _ 14 NMOD 14 group _ NN NN NOUN _ 15 SUB 15 could _ MD MD VERB _ 11 SBAR 16 n't _ RB RB ADV _ 15 VMOD 17 get _ VB VB VERB _ 15 VC 18 financing _ NN NN NOUN _ 17 OBJ 19 for _ IN IN ADP _ 18 NMOD 20 its _ PR PRP$ PRON _ 21 NMOD 21 bid _ NN NN NOUN _ 19 PMOD 22 . _ . . . _ 9 P 1 At _ IN IN ADP _ 7 VMOD 2 this _ DT DT DET _ 3 NMOD 3 point _ NN NN NOUN _ 1 PMOD 4 , _ , , . _ 7 P 5 the _ DT DT DET _ 6 NMOD 6 Dow _ NN NNP NOUN _ 7 SUB 7 was _ VB VBD VERB _ 0 ROOT 8 down _ RB RB ADV _ 7 PRD 9 about _ RB RB ADV _ 11 NMOD 10 35 _ CD CD NUM _ 9 AMOD 11 points _ NN NNS NOUN _ 8 AMOD 12 . _ . . . _ 7 P 1 The _ DT DT DET _ 2 NMOD 2 market _ NN NN NOUN _ 3 SUB 3 crumbled _ VB VBD VERB _ 0 ROOT 4 . _ . . . _ 3 P 1 Arbitragers _ NN NNS NOUN _ 2 SUB 2 could _ MD MD VERB _ 11 VMOD 3 n't _ RB RB ADV _ 2 VMOD 4 dump _ VB VB VERB _ 2 VC 5 their _ PR PRP$ PRON _ 7 NMOD 6 UAL _ NN NNP NOUN _ 7 NMOD 7 stock _ NN NN NOUN _ 4 OBJ 8 -- _ : : . _ 11 P 9 but _ CC CC CONJ _ 11 VMOD 10 they _ PR PRP PRON _ 11 SUB 11 rid _ VB VBD VERB _ 0 ROOT 12 themselves _ PR PRP PRON _ 11 OBJ 13 of _ IN IN ADP _ 11 VMOD 14 nearly _ RB RB ADV _ 15 AMOD 15 every _ DT DT DET _ 19 NMOD 16 `` _ `` `` . _ 19 P 17 rumor _ NN NN NOUN _ 19 NMOD 18 '' _ '' '' . _ 19 P 19 stock _ NN NN NOUN _ 13 PMOD 20 they _ PR PRP PRON _ 21 SUB 21 had _ VB VBD VERB _ 19 NMOD 22 . _ . . . _ 11 P 1 For _ IN IN ADP _ 6 VMOD 2 example _ NN NN NOUN _ 1 PMOD 3 , _ , , . _ 6 P 4 their _ PR PRP$ PRON _ 5 NMOD 5 selling _ NN NN NOUN _ 6 SUB 6 caused _ VB VBD VERB _ 0 ROOT 7 trading _ NN NN NOUN _ 8 NMOD 8 halts _ NN NNS NOUN _ 10 SUB 9 to _ TO TO PRT _ 10 VMOD 10 be _ VB VB VERB _ 6 VMOD 11 declared _ VB VBN VERB _ 10 VC 12 in _ IN IN ADP _ 11 VMOD 13 USAir _ NN NNP NOUN _ 14 NMOD 14 Group _ NN NNP NOUN _ 39 NMOD 15 , _ , , . _ 14 P 16 which _ WD WDT DET _ 14 NMOD 17 closed _ VB VBD VERB _ 16 SBAR 18 down _ RB RB ADV _ 17 VMOD 19 3 _ CD CD NUM _ 20 AMOD 20 7\/8 _ CD CD NUM _ 18 AMOD 21 to _ TO TO PRT _ 18 AMOD 22 41 _ CD CD NUM _ 23 AMOD 23 1\/2 _ CD CD NUM _ 21 PMOD 24 , _ , , . _ 39 P 25 Delta _ NN NNP NOUN _ 27 NMOD 26 Air _ NN NNP NOUN _ 27 NMOD 27 Lines _ NN NNP NOUN _ 39 NMOD 28 , _ , , . _ 27 P 29 which _ WD WDT DET _ 27 NMOD 30 fell _ VB VBD VERB _ 29 SBAR 31 7 _ CD CD NUM _ 32 AMOD 32 3\/4 _ CD CD NUM _ 30 VMOD 33 to _ TO TO PRT _ 30 VMOD 34 69 _ CD CD NUM _ 35 AMOD 35 1\/4 _ CD CD NUM _ 33 PMOD 36 , _ , , . _ 39 P 37 and _ CC CC CONJ _ 39 NMOD 38 Philips _ NN NNP NOUN _ 39 NMOD 39 Industries _ NN NNP NOUN _ 12 PMOD 40 , _ , , . _ 39 P 41 which _ WD WDT DET _ 39 NMOD 42 sank _ VB VBD VERB _ 41 SBAR 43 3 _ CD CD NUM _ 42 VMOD 44 to _ TO TO PRT _ 42 VMOD 45 21 _ CD CD NUM _ 46 AMOD 46 1\/2 _ CD CD NUM _ 44 PMOD 47 . _ . . . _ 6 P 1 These _ DT DT DET _ 2 NMOD 2 stocks _ NN NNS NOUN _ 4 SUB 3 eventually _ RB RB ADV _ 4 VMOD 4 reopened _ VB VBD VERB _ 0 ROOT 5 . _ . . . _ 4 P 1 But _ CC CC CONJ _ 7 VMOD 2 as _ IN IN ADP _ 7 VMOD 3 panic _ NN NN NOUN _ 4 SUB 4 spread _ VB VBD VERB _ 2 SBAR 5 , _ , , . _ 7 P 6 speculators _ NN NNS NOUN _ 7 SUB 7 began _ VB VBD VERB _ 0 ROOT 8 to _ TO TO PRT _ 9 VMOD 9 sell _ VB VB VERB _ 7 VMOD 10 blue-chip _ JJ JJ ADJ _ 11 NMOD 11 stocks _ NN NNS NOUN _ 9 OBJ 12 such _ JJ JJ ADJ _ 13 PMOD 13 as _ IN IN ADP _ 11 NMOD 14 Philip _ NN NNP NOUN _ 15 NMOD 15 Morris _ NN NNP NOUN _ 19 NMOD 16 and _ CC CC CONJ _ 19 NMOD 17 International _ NN NNP NOUN _ 19 NMOD 18 Business _ NN NNP NOUN _ 19 NMOD 19 Machines _ NN NNP NOUN _ 13 PMOD 20 to _ TO TO PRT _ 21 VMOD 21 offset _ VB VB VERB _ 9 VMOD 22 their _ PR PRP$ PRON _ 23 NMOD 23 losses _ NN NNS NOUN _ 21 OBJ 24 . _ . . . _ 7 P 1 When _ WR WRB ADV _ 11 VMOD 2 trading _ NN NN NOUN _ 3 SUB 3 was _ VB VBD VERB _ 1 SBAR 4 halted _ VB VBN VERB _ 3 VC 5 in _ IN IN ADP _ 4 VMOD 6 Philip _ NN NNP NOUN _ 7 NMOD 7 Morris _ NN NNP NOUN _ 5 PMOD 8 , _ , , . _ 11 P 9 the _ DT DT DET _ 10 NMOD 10 stock _ NN NN NOUN _ 11 SUB 11 was _ VB VBD VERB _ 0 ROOT 12 trading _ VB VBG VERB _ 11 VC 13 at _ IN IN ADP _ 12 VMOD 14 41 _ CD CD NUM _ 13 PMOD 15 , _ , , . _ 12 P 16 down _ RB RB ADV _ 12 VMOD 17 3 _ CD CD NUM _ 18 AMOD 18 3\/8 _ CD CD NUM _ 16 AMOD 19 , _ , , . _ 12 P 20 while _ IN IN ADP _ 12 VMOD 21 IBM _ NN NNP NOUN _ 22 SUB 22 closed _ VB VBD VERB _ 20 SBAR 23 5 _ CD CD NUM _ 24 AMOD 24 5\/8 _ CD CD NUM _ 25 AMOD 25 lower _ JJ JJR ADJ _ 22 VMOD 26 at _ IN IN ADP _ 22 VMOD 27 102 _ CD CD NUM _ 26 PMOD 28 . _ . . . _ 11 P 1 Selling _ NN NN NOUN _ 2 SUB 2 snowballed _ VB VBD VERB _ 0 ROOT 3 because _ IN IN ADP _ 2 VMOD 4 of _ IN IN ADP _ 3 PMOD 5 waves _ NN NNS NOUN _ 3 PMOD 6 of _ IN IN ADP _ 5 NMOD 7 automatic _ JJ JJ ADJ _ 11 NMOD 8 `` _ `` `` . _ 11 P 9 stop-loss _ JJ JJ ADJ _ 11 NMOD 10 '' _ '' '' . _ 11 P 11 orders _ NN NNS NOUN _ 6 PMOD 12 , _ , , . _ 11 P 13 which _ WD WDT DET _ 11 NMOD 14 are _ VB VBP VERB _ 13 SBAR 15 triggered _ VB VBN VERB _ 14 VC 16 by _ IN IN ADP _ 15 VMOD 17 computer _ NN NN NOUN _ 16 PMOD 18 when _ WR WRB ADV _ 15 VMOD 19 prices _ NN NNS NOUN _ 20 SUB 20 fall _ VB VBP VERB _ 18 SBAR 21 to _ TO TO PRT _ 20 VMOD 22 certain _ JJ JJ ADJ _ 23 NMOD 23 levels _ NN NNS NOUN _ 21 PMOD 24 . _ . . . _ 2 P 1 Most _ JJ JJS ADJ _ 7 SUB 2 of _ IN IN ADP _ 1 NMOD 3 the _ DT DT DET _ 6 NMOD 4 stock _ NN NN NOUN _ 6 NMOD 5 selling _ NN NN NOUN _ 6 NMOD 6 pressure _ NN NN NOUN _ 2 PMOD 7 came _ VB VBD VERB _ 0 ROOT 8 from _ IN IN ADP _ 7 VMOD 9 Wall _ NN NNP NOUN _ 11 NMOD 10 Street _ NN NNP NOUN _ 11 NMOD 11 professionals _ NN NNS NOUN _ 8 PMOD 12 , _ , , . _ 11 P 13 including _ VB VBG VERB _ 11 NMOD 14 computer-guided _ JJ JJ ADJ _ 16 NMOD 15 program _ NN NN NOUN _ 16 NMOD 16 traders _ NN NNS NOUN _ 13 PMOD 17 . _ . . . _ 7 P 1 Traders _ NN NNS NOUN _ 2 SUB 2 said _ VB VBD VERB _ 0 ROOT 3 most _ JJ JJS ADJ _ 15 SUB 4 of _ IN IN ADP _ 3 NMOD 5 their _ PR PRP$ PRON _ 8 NMOD 6 major _ JJ JJ ADJ _ 8 NMOD 7 institutional _ JJ JJ ADJ _ 8 NMOD 8 investors _ NN NNS NOUN _ 4 PMOD 9 , _ , , . _ 15 P 10 on _ IN IN ADP _ 15 VMOD 11 the _ DT DT DET _ 13 NMOD 12 other _ JJ JJ ADJ _ 13 NMOD 13 hand _ NN NN NOUN _ 10 PMOD 14 , _ , , . _ 15 P 15 sat _ VB VBD VERB _ 2 VMOD 16 tight _ RB RB ADV _ 15 VMOD 17 . _ . . . _ 2 P 1 Now _ RB RB ADV _ 15 VMOD 2 , _ , , . _ 15 P 3 at _ IN IN ADP _ 15 VMOD 4 3:07 _ CD CD NUM _ 3 PMOD 5 , _ , , . _ 15 P 6 one _ CD CD NUM _ 15 SUB 7 of _ IN IN ADP _ 6 NMOD 8 the _ DT DT DET _ 10 NMOD 9 market _ NN NN NOUN _ 10 NMOD 10 's _ PO POS PRT _ 13 NMOD 11 post-crash _ JJ JJ ADJ _ 13 NMOD 12 `` _ `` `` . _ 13 P 13 reforms _ NN NNS NOUN _ 7 PMOD 14 '' _ '' '' . _ 13 P 15 took _ VB VBD VERB _ 0 ROOT 16 hold _ NN NN NOUN _ 15 OBJ 17 as _ IN IN ADP _ 15 VMOD 18 the _ DT DT DET _ 22 NMOD 19 S&P _ NN NNP NOUN _ 22 NMOD 20 500 _ CD CD NUM _ 22 NMOD 21 futures _ NN NNS NOUN _ 22 NMOD 22 contract _ NN NN NOUN _ 23 SUB 23 had _ VB VBD VERB _ 17 SBAR 24 plunged _ VB VBN VERB _ 23 VC 25 12 _ CD CD NUM _ 26 NMOD 26 points _ NN NNS NOUN _ 24 VMOD 27 , _ , , . _ 26 P 28 equivalent _ JJ JJ ADJ _ 26 NMOD 29 to _ TO TO PRT _ 28 AMOD 30 around _ IN IN ADP _ 33 NMOD 31 a _ DT DT DET _ 30 AMOD 32 100-point _ JJ JJ ADJ _ 30 AMOD 33 drop _ NN NN NOUN _ 29 PMOD 34 in _ IN IN ADP _ 33 NMOD 35 the _ DT DT DET _ 37 NMOD 36 Dow _ NN NNP NOUN _ 37 NMOD 37 industrials _ NN NNS NOUN _ 34 PMOD 38 . _ . . . _ 15 P 1 Under _ IN IN ADP _ 16 VMOD 2 an _ DT DT DET _ 3 NMOD 3 agreement _ NN NN NOUN _ 1 PMOD 4 signed _ VB VBN VERB _ 3 NMOD 5 by _ IN IN ADP _ 4 VMOD 6 the _ DT DT DET _ 8 NMOD 7 Big _ NN NNP NOUN _ 8 NMOD 8 Board _ NN NNP NOUN _ 13 NMOD 9 and _ CC CC CONJ _ 13 NMOD 10 the _ DT DT DET _ 13 NMOD 11 Chicago _ NN NNP NOUN _ 13 NMOD 12 Mercantile _ NN NNP NOUN _ 13 NMOD 13 Exchange _ NN NNP NOUN _ 5 PMOD 14 , _ , , . _ 16 P 15 trading _ NN NN NOUN _ 16 SUB 16 was _ VB VBD VERB _ 0 ROOT 17 temporarily _ RB RB ADV _ 16 VMOD 18 halted _ VB VBN VERB _ 16 VC 19 in _ IN IN ADP _ 18 VMOD 20 Chicago _ NN NNP NOUN _ 19 PMOD 21 . _ . . . _ 16 P 1 After _ IN IN ADP _ 28 VMOD 2 the _ DT DT DET _ 4 NMOD 3 trading _ NN NN NOUN _ 4 NMOD 4 halt _ NN NN NOUN _ 1 PMOD 5 in _ IN IN ADP _ 4 NMOD 6 the _ DT DT DET _ 9 NMOD 7 S&P _ NN NNP NOUN _ 9 NMOD 8 500 _ CD CD NUM _ 9 NMOD 9 pit _ NN NN NOUN _ 5 PMOD 10 in _ IN IN ADP _ 9 NMOD 11 Chicago _ NN NNP NOUN _ 10 PMOD 12 , _ , , . _ 28 P 13 waves _ NN NNS NOUN _ 16 SUB 14 of _ IN IN ADP _ 13 NMOD 15 selling _ NN NN NOUN _ 14 PMOD 16 continued _ VB VBD VERB _ 28 VMOD 17 to _ TO TO PRT _ 18 VMOD 18 hit _ VB VB VERB _ 16 VMOD 19 stocks _ NN NNS NOUN _ 20 NMOD 20 themselves _ PR PRP PRON _ 18 OBJ 21 on _ IN IN ADP _ 18 VMOD 22 the _ DT DT DET _ 24 NMOD 23 Big _ NN NNP NOUN _ 24 NMOD 24 Board _ NN NNP NOUN _ 21 PMOD 25 , _ , , . _ 28 P 26 and _ CC CC CONJ _ 28 VMOD 27 specialists _ NN NNS NOUN _ 28 SUB 28 continued _ VB VBD VERB _ 0 ROOT 29 to _ TO TO PRT _ 30 VMOD 30 notch _ VB VB VERB _ 28 VMOD 31 prices _ NN NNS NOUN _ 30 OBJ 32 down _ RP RP PRT _ 30 VMOD 33 . _ . . . _ 28 P 1 As _ IN IN ADP _ 13 VMOD 2 a _ DT DT DET _ 3 NMOD 3 result _ NN NN NOUN _ 1 PMOD 4 , _ , , . _ 13 P 5 the _ DT DT DET _ 6 NMOD 6 link _ NN NN NOUN _ 13 SUB 7 between _ IN IN ADP _ 6 NMOD 8 the _ DT DT DET _ 12 NMOD 9 futures _ NN NNS NOUN _ 12 NMOD 10 and _ CC CC CONJ _ 12 NMOD 11 stock _ NN NN NOUN _ 12 NMOD 12 markets _ NN NNS NOUN _ 7 PMOD 13 ripped _ VB VBD VERB _ 0 ROOT 14 apart _ RB RB ADV _ 13 VMOD 15 . _ . . . _ 13 P vowpal-wabbit-8.5.0.dfsg1/demo/dependencyparsing/wsj_train_subset000066400000000000000000002542701323713302100252500ustar00rootroot000000000000001 In _ IN IN ADP _ 43 VMOD 2 an _ DT DT DET _ 5 NMOD 3 Oct. _ NN NNP NOUN _ 5 NMOD 4 19 _ CD CD NUM _ 5 NMOD 5 review _ NN NN NOUN _ 1 PMOD 6 of _ IN IN ADP _ 5 NMOD 7 `` _ `` `` . _ 9 P 8 The _ DT DT DET _ 9 NMOD 9 Misanthrope _ NN NN NOUN _ 6 PMOD 10 '' _ '' '' . _ 9 P 11 at _ IN IN ADP _ 9 NMOD 12 Chicago _ NN NNP NOUN _ 13 NMOD 13 's _ PO POS PRT _ 15 NMOD 14 Goodman _ NN NNP NOUN _ 15 NMOD 15 Theatre _ NN NNP NOUN _ 11 PMOD 16 ( _ -L -LRB- . _ 30 P 17 `` _ `` `` . _ 30 P 18 Revitalized _ VB VBN VERB _ 19 NMOD 19 Classics _ NN NNS NOUN _ 20 SUB 20 Take _ VB VBP VERB _ 30 DEP 21 the _ DT DT DET _ 22 NMOD 22 Stage _ NN NN NOUN _ 20 OBJ 23 in _ IN IN ADP _ 20 VMOD 24 Windy _ NN NNP NOUN _ 25 NMOD 25 City _ NN NNP NOUN _ 23 PMOD 26 , _ , , . _ 30 P 27 '' _ '' '' . _ 30 P 28 Leisure _ NN NN NOUN _ 30 NMOD 29 & _ CC CC CONJ _ 30 NMOD 30 Arts _ NN NNS NOUN _ 5 NMOD 31 ) _ -R -RRB- . _ 30 P 32 , _ , , . _ 43 P 33 the _ DT DT DET _ 34 NMOD 34 role _ NN NN NOUN _ 43 SUB 35 of _ IN IN ADP _ 34 NMOD 36 Celimene _ NN NNP NOUN _ 35 PMOD 37 , _ , , . _ 34 P 38 played _ VB VBN VERB _ 34 NMOD 39 by _ IN IN ADP _ 38 VMOD 40 Kim _ NN NNP NOUN _ 41 NMOD 41 Cattrall _ NN NNP NOUN _ 39 PMOD 42 , _ , , . _ 34 P 43 was _ VB VBD VERB _ 0 ROOT 44 mistakenly _ RB RB ADV _ 45 VMOD 45 attributed _ VB VBN VERB _ 43 VC 46 to _ TO TO PRT _ 45 VMOD 47 Christina _ NN NNP NOUN _ 48 NMOD 48 Haag _ NN NNP NOUN _ 46 PMOD 49 . _ . . . _ 43 P 1 Ms. _ NN NNP NOUN _ 2 NMOD 2 Haag _ NN NNP NOUN _ 3 SUB 3 plays _ VB VBZ VERB _ 0 ROOT 4 Elianti _ NN NNP NOUN _ 3 OBJ 5 . _ . . . _ 3 P 1 Rolls-Royce _ NN NNP NOUN _ 4 NMOD 2 Motor _ NN NNP NOUN _ 4 NMOD 3 Cars _ NN NNPS NOUN _ 4 NMOD 4 Inc. _ NN NNP NOUN _ 5 SUB 5 said _ VB VBD VERB _ 0 ROOT 6 it _ PR PRP PRON _ 7 SUB 7 expects _ VB VBZ VERB _ 5 VMOD 8 its _ PR PRP$ PRON _ 10 NMOD 9 U.S. _ NN NNP NOUN _ 10 NMOD 10 sales _ NN NNS NOUN _ 12 SUB 11 to _ TO TO PRT _ 12 VMOD 12 remain _ VB VB VERB _ 7 VMOD 13 steady _ JJ JJ ADJ _ 12 PRD 14 at _ IN IN ADP _ 12 VMOD 15 about _ IN IN ADP _ 17 NMOD 16 1,200 _ CD CD NUM _ 15 AMOD 17 cars _ NN NNS NOUN _ 14 PMOD 18 in _ IN IN ADP _ 12 VMOD 19 1990 _ CD CD NUM _ 18 PMOD 20 . _ . . . _ 5 P 1 The _ DT DT DET _ 4 NMOD 2 luxury _ NN NN NOUN _ 4 NMOD 3 auto _ NN NN NOUN _ 4 NMOD 4 maker _ NN NN NOUN _ 7 SUB 5 last _ JJ JJ ADJ _ 6 NMOD 6 year _ NN NN NOUN _ 7 VMOD 7 sold _ VB VBD VERB _ 0 ROOT 8 1,214 _ CD CD NUM _ 9 NMOD 9 cars _ NN NNS NOUN _ 7 OBJ 10 in _ IN IN ADP _ 7 VMOD 11 the _ DT DT DET _ 12 NMOD 12 U.S. _ NN NNP NOUN _ 10 PMOD 1 Howard _ NN NNP NOUN _ 2 NMOD 2 Mosher _ NN NNP NOUN _ 8 NMOD 3 , _ , , . _ 8 P 4 president _ NN NN NOUN _ 8 NMOD 5 and _ CC CC CONJ _ 8 NMOD 6 chief _ JJ JJ ADJ _ 8 NMOD 7 executive _ NN NN NOUN _ 8 NMOD 8 officer _ NN NN NOUN _ 10 SUB 9 , _ , , . _ 8 P 10 said _ VB VBD VERB _ 0 ROOT 11 he _ PR PRP PRON _ 12 SUB 12 anticipates _ VB VBZ VERB _ 10 VMOD 13 growth _ NN NN NOUN _ 12 OBJ 14 for _ IN IN ADP _ 13 NMOD 15 the _ DT DT DET _ 18 NMOD 16 luxury _ NN NN NOUN _ 18 NMOD 17 auto _ NN NN NOUN _ 18 NMOD 18 maker _ NN NN NOUN _ 14 PMOD 19 in _ IN IN ADP _ 13 NMOD 20 Britain _ NN NNP NOUN _ 22 NMOD 21 and _ CC CC CONJ _ 22 NMOD 22 Europe _ NN NNP NOUN _ 19 PMOD 23 , _ , , . _ 19 P 24 and _ CC CC CONJ _ 19 PMOD 25 in _ IN IN ADP _ 19 PMOD 26 Far _ JJ JJ ADJ _ 27 AMOD 27 Eastern _ JJ JJ ADJ _ 28 NMOD 28 markets _ NN NNS NOUN _ 25 PMOD 29 . _ . . . _ 10 P 1 BELL _ NN NNP NOUN _ 3 NMOD 2 INDUSTRIES _ NN NNP NOUN _ 3 NMOD 3 Inc. _ NN NNP NOUN _ 4 SUB 4 increased _ VB VBD VERB _ 0 ROOT 5 its _ PR PRP$ PRON _ 6 NMOD 6 quarterly _ NN NN NOUN _ 4 OBJ 7 to _ TO TO PRT _ 4 VMOD 8 10 _ CD CD NUM _ 9 NMOD 9 cents _ NN NNS NOUN _ 7 PMOD 10 from _ IN IN ADP _ 4 VMOD 11 seven _ CD CD NUM _ 12 NMOD 12 cents _ NN NNS NOUN _ 14 NMOD 13 a _ DT DT DET _ 14 NMOD 14 share _ NN NN NOUN _ 10 PMOD 15 . _ . . . _ 4 P 1 The _ DT DT DET _ 3 NMOD 2 new _ JJ JJ ADJ _ 3 NMOD 3 rate _ NN NN NOUN _ 4 SUB 4 will _ MD MD VERB _ 0 ROOT 5 be _ VB VB VERB _ 4 VC 6 payable _ JJ JJ ADJ _ 5 PRD 7 Feb. _ NN NNP NOUN _ 6 AMOD 8 15 _ CD CD NUM _ 7 NMOD 9 . _ . . . _ 4 P 1 A _ DT DT DET _ 3 NMOD 2 record _ NN NN NOUN _ 3 NMOD 3 date _ NN NN NOUN _ 4 SUB 4 has _ VB VBZ VERB _ 0 ROOT 5 n't _ RB RB ADV _ 4 VMOD 6 been _ VB VBN VERB _ 4 VC 7 set _ VB VBN VERB _ 6 VC 8 . _ . . . _ 4 P 1 Bell _ NN NNP NOUN _ 8 SUB 2 , _ , , . _ 1 P 3 based _ VB VBN VERB _ 1 NMOD 4 in _ IN IN ADP _ 3 VMOD 5 Los _ NN NNP NOUN _ 6 NMOD 6 Angeles _ NN NNP NOUN _ 4 PMOD 7 , _ , , . _ 1 P 8 makes _ VB VBZ VERB _ 0 ROOT 9 and _ CC CC CONJ _ 8 VMOD 10 distributes _ VB VBZ VERB _ 8 VMOD 11 electronic _ JJ JJ ADJ _ 16 NMOD 12 , _ , , . _ 11 P 13 computer _ NN NN NOUN _ 11 DEP 14 and _ CC CC CONJ _ 11 DEP 15 building _ NN NN NOUN _ 11 DEP 16 products _ NN NNS NOUN _ 8 OBJ 17 . _ . . . _ 8 P 1 Investors _ NN NNS NOUN _ 2 SUB 2 are _ VB VBP VERB _ 0 ROOT 3 appealing _ VB VBG VERB _ 2 VC 4 to _ TO TO PRT _ 3 VMOD 5 the _ DT DT DET _ 9 NMOD 6 Securities _ NN NNPS NOUN _ 9 NMOD 7 and _ CC CC CONJ _ 9 NMOD 8 Exchange _ NN NNP NOUN _ 9 NMOD 9 Commission _ NN NNP NOUN _ 4 PMOD 10 not _ RB RB ADV _ 12 VMOD 11 to _ TO TO PRT _ 12 VMOD 12 limit _ VB VB VERB _ 3 VMOD 13 their _ PR PRP$ PRON _ 14 NMOD 14 access _ NN NN NOUN _ 12 OBJ 15 to _ TO TO PRT _ 14 NMOD 16 information _ NN NN NOUN _ 15 PMOD 17 about _ IN IN ADP _ 16 NMOD 18 stock _ NN NN NOUN _ 21 NMOD 19 purchases _ NN NNS NOUN _ 21 NMOD 20 and _ CC CC CONJ _ 21 NMOD 21 sales _ NN NNS NOUN _ 17 PMOD 22 by _ IN IN ADP _ 21 NMOD 23 corporate _ JJ JJ ADJ _ 24 NMOD 24 insiders _ NN NNS NOUN _ 22 PMOD 25 . _ . . . _ 2 P 1 A _ DT DT DET _ 3 NMOD 2 SEC _ NN NNP NOUN _ 3 NMOD 3 proposal _ NN NN NOUN _ 12 SUB 4 to _ TO TO PRT _ 5 VMOD 5 ease _ VB VB VERB _ 3 NMOD 6 reporting _ NN NN NOUN _ 7 NMOD 7 requirements _ NN NNS NOUN _ 5 OBJ 8 for _ IN IN ADP _ 7 NMOD 9 some _ DT DT DET _ 11 NMOD 10 company _ NN NN NOUN _ 11 NMOD 11 executives _ NN NNS NOUN _ 8 PMOD 12 would _ MD MD VERB _ 32 VMOD 13 undermine _ VB VB VERB _ 12 VC 14 the _ DT DT DET _ 15 NMOD 15 usefulness _ NN NN NOUN _ 13 OBJ 16 of _ IN IN ADP _ 15 NMOD 17 information _ NN NN NOUN _ 16 PMOD 18 on _ IN IN ADP _ 17 NMOD 19 insider _ NN NN NOUN _ 20 NMOD 20 trades _ NN NNS NOUN _ 18 PMOD 21 as _ IN IN ADP _ 15 NMOD 22 a _ DT DT DET _ 24 NMOD 23 stock-picking _ JJ JJ ADJ _ 24 NMOD 24 tool _ NN NN NOUN _ 21 PMOD 25 , _ , , . _ 32 P 26 individual _ JJ JJ ADJ _ 27 NMOD 27 investors _ NN NNS NOUN _ 31 NMOD 28 and _ CC CC CONJ _ 31 NMOD 29 professional _ JJ JJ ADJ _ 31 NMOD 30 money _ NN NN NOUN _ 31 NMOD 31 managers _ NN NNS NOUN _ 32 SUB 32 contend _ VB VBP VERB _ 0 ROOT 33 . _ . . . _ 32 P 1 They _ PR PRP PRON _ 2 SUB 2 make _ VB VBP VERB _ 0 ROOT 3 the _ DT DT DET _ 4 NMOD 4 argument _ NN NN NOUN _ 2 OBJ 5 in _ IN IN ADP _ 2 VMOD 6 letters _ NN NNS NOUN _ 5 PMOD 7 to _ TO TO PRT _ 6 NMOD 8 the _ DT DT DET _ 9 NMOD 9 agency _ NN NN NOUN _ 7 PMOD 10 about _ IN IN ADP _ 6 NMOD 11 rule _ NN NN NOUN _ 12 NMOD 12 changes _ NN NNS NOUN _ 10 PMOD 13 proposed _ VB VBD VERB _ 12 NMOD 14 this _ DT DT DET _ 16 NMOD 15 past _ JJ JJ ADJ _ 16 NMOD 16 summer _ NN NN NOUN _ 13 VMOD 17 that _ IN IN ADP _ 12 NMOD 18 , _ , , . _ 17 P 19 among _ IN IN ADP _ 23 VMOD 20 other _ JJ JJ ADJ _ 21 NMOD 21 things _ NN NNS NOUN _ 19 PMOD 22 , _ , , . _ 23 P 23 would _ MD MD VERB _ 17 SBAR 24 exempt _ VB VB VERB _ 23 VC 25 many _ JJ JJ ADJ _ 27 NMOD 26 middle-management _ JJ JJ ADJ _ 27 NMOD 27 executives _ NN NNS NOUN _ 24 OBJ 28 from _ IN IN ADP _ 24 VMOD 29 reporting _ VB VBG VERB _ 28 PMOD 30 trades _ NN NNS NOUN _ 29 OBJ 31 in _ IN IN ADP _ 30 NMOD 32 their _ PR PRP$ PRON _ 35 NMOD 33 own _ JJ JJ ADJ _ 35 NMOD 34 companies _ NN NNS NOUN _ 35 NMOD 35 ' _ PO POS PRT _ 36 NMOD 36 shares _ NN NNS NOUN _ 31 PMOD 37 . _ . . . _ 2 P 1 The _ DT DT DET _ 3 NMOD 2 proposed _ VB VBN VERB _ 3 NMOD 3 changes _ NN NNS NOUN _ 5 SUB 4 also _ RB RB ADV _ 5 VMOD 5 would _ MD MD VERB _ 0 ROOT 6 allow _ VB VB VERB _ 5 VC 7 executives _ NN NNS NOUN _ 9 SUB 8 to _ TO TO PRT _ 9 VMOD 9 report _ VB VB VERB _ 6 VMOD 10 exercises _ NN NNS NOUN _ 9 OBJ 11 of _ IN IN ADP _ 10 NMOD 12 options _ NN NNS NOUN _ 11 PMOD 13 later _ RB RBR ADV _ 9 VMOD 14 and _ CC CC CONJ _ 13 AMOD 15 less _ RB RBR ADV _ 16 AMOD 16 often _ RB RB ADV _ 13 AMOD 17 . _ . . . _ 5 P 1 Many _ JJ JJ ADJ _ 5 SUB 2 of _ IN IN ADP _ 1 NMOD 3 the _ DT DT DET _ 4 NMOD 4 letters _ NN NNS NOUN _ 2 PMOD 5 maintain _ VB VBP VERB _ 0 ROOT 6 that _ IN IN ADP _ 5 VMOD 7 investor _ NN NN NOUN _ 8 NMOD 8 confidence _ NN NN NOUN _ 9 SUB 9 has _ VB VBZ VERB _ 25 VMOD 10 been _ VB VBN VERB _ 9 VC 11 so _ RB RB ADV _ 12 VMOD 12 shaken _ VB VBN VERB _ 10 VC 13 by _ IN IN ADP _ 12 VMOD 14 the _ DT DT DET _ 18 NMOD 15 1987 _ CD CD NUM _ 18 NMOD 16 stock _ NN NN NOUN _ 18 NMOD 17 market _ NN NN NOUN _ 18 NMOD 18 crash _ NN NN NOUN _ 13 PMOD 19 -- _ : : . _ 25 P 20 and _ CC CC CONJ _ 25 VMOD 21 the _ DT DT DET _ 22 NMOD 22 markets _ NN NNS NOUN _ 25 SUB 23 already _ RB RB ADV _ 25 VMOD 24 so _ RB RB ADV _ 25 VMOD 25 stacked _ VB VBN VERB _ 6 SBAR 26 against _ IN IN ADP _ 25 VMOD 27 the _ DT DT DET _ 29 NMOD 28 little _ JJ JJ ADJ _ 29 NMOD 29 guy _ NN NN NOUN _ 26 PMOD 30 -- _ : : . _ 25 P 31 that _ IN IN ADP _ 25 VMOD 32 any _ DT DT DET _ 33 NMOD 33 decrease _ NN NN NOUN _ 39 SUB 34 in _ IN IN ADP _ 33 NMOD 35 information _ NN NN NOUN _ 34 PMOD 36 on _ IN IN ADP _ 35 NMOD 37 insider-trading _ NN NN NOUN _ 38 NMOD 38 patterns _ NN NNS NOUN _ 36 PMOD 39 might _ MD MD VERB _ 31 SBAR 40 prompt _ VB VB VERB _ 39 VC 41 individuals _ NN NNS NOUN _ 43 SUB 42 to _ TO TO PRT _ 43 VMOD 43 get _ VB VB VERB _ 40 VMOD 44 out _ RB RB ADV _ 43 VMOD 45 of _ IN IN ADP _ 44 AMOD 46 stocks _ NN NNS NOUN _ 45 PMOD 47 altogether _ RB RB ADV _ 43 VMOD 48 . _ . . . _ 5 P 1 `` _ `` `` . _ 18 P 2 The _ DT DT DET _ 3 NMOD 3 SEC _ NN NNP NOUN _ 4 SUB 4 has _ VB VBZ VERB _ 18 VMOD 5 historically _ RB RB ADV _ 4 VMOD 6 paid _ VB VBN VERB _ 4 VC 7 obeisance _ NN NN NOUN _ 6 OBJ 8 to _ TO TO PRT _ 6 VMOD 9 the _ DT DT DET _ 10 NMOD 10 ideal _ NN NN NOUN _ 8 PMOD 11 of _ IN IN ADP _ 10 NMOD 12 a _ DT DT DET _ 15 NMOD 13 level _ JJ JJ ADJ _ 15 NMOD 14 playing _ NN NN NOUN _ 15 NMOD 15 field _ NN NN NOUN _ 11 PMOD 16 , _ , , . _ 18 P 17 '' _ '' '' . _ 18 P 18 wrote _ VB VBD VERB _ 0 ROOT 19 Clyde _ NN NNP NOUN _ 21 NMOD 20 S. _ NN NNP NOUN _ 21 NMOD 21 McGregor _ NN NNP NOUN _ 18 SUB 22 of _ IN IN ADP _ 21 NMOD 23 Winnetka _ NN NNP NOUN _ 25 NMOD 24 , _ , , . _ 25 P 25 Ill. _ NN NNP NOUN _ 22 PMOD 26 , _ , , . _ 25 P 27 in _ IN IN ADP _ 18 VMOD 28 one _ CD CD NUM _ 27 PMOD 29 of _ IN IN ADP _ 28 NMOD 30 the _ DT DT DET _ 32 NMOD 31 92 _ CD CD NUM _ 32 NMOD 32 letters _ NN NNS NOUN _ 29 PMOD 33 the _ DT DT DET _ 34 NMOD 34 agency _ NN NN NOUN _ 35 SUB 35 has _ VB VBZ VERB _ 32 NMOD 36 received _ VB VBN VERB _ 35 VC 37 since _ IN IN ADP _ 36 VMOD 38 the _ DT DT DET _ 39 NMOD 39 changes _ NN NNS NOUN _ 40 SUB 40 were _ VB VBD VERB _ 37 SBAR 41 proposed _ VB VBN VERB _ 40 VC 42 Aug. _ NN NNP NOUN _ 41 VMOD 43 17 _ CD CD NUM _ 42 NMOD 44 . _ . . . _ 18 P 1 `` _ `` `` . _ 5 P 2 Apparently _ RB RB ADV _ 5 VMOD 3 the _ DT DT DET _ 4 NMOD 4 commission _ NN NN NOUN _ 5 SUB 5 did _ VB VBD VERB _ 0 ROOT 6 not _ RB RB ADV _ 5 VMOD 7 really _ RB RB ADV _ 5 VMOD 8 believe _ VB VB VERB _ 5 VC 9 in _ IN IN ADP _ 8 VMOD 10 this _ DT DT DET _ 11 NMOD 11 ideal _ NN NN NOUN _ 9 PMOD 12 . _ . . . _ 5 P 13 '' _ '' '' . _ 5 P 1 Currently _ RB RB ADV _ 5 VMOD 2 , _ , , . _ 5 P 3 the _ DT DT DET _ 4 NMOD 4 rules _ NN NNS NOUN _ 5 SUB 5 force _ VB VBP VERB _ 0 ROOT 6 executives _ NN NNS NOUN _ 12 NMOD 7 , _ , , . _ 12 P 8 directors _ NN NNS NOUN _ 12 NMOD 9 and _ CC CC CONJ _ 12 NMOD 10 other _ JJ JJ ADJ _ 12 NMOD 11 corporate _ JJ JJ ADJ _ 12 NMOD 12 insiders _ NN NNS NOUN _ 14 SUB 13 to _ TO TO PRT _ 14 VMOD 14 report _ VB VB VERB _ 5 VMOD 15 purchases _ NN NNS NOUN _ 17 NMOD 16 and _ CC CC CONJ _ 17 NMOD 17 sales _ NN NNS NOUN _ 14 OBJ 18 of _ IN IN ADP _ 17 NMOD 19 their _ PR PRP$ PRON _ 21 NMOD 20 companies _ NN NNS NOUN _ 21 NMOD 21 ' _ PO POS PRT _ 22 NMOD 22 shares _ NN NNS NOUN _ 18 PMOD 23 within _ IN IN ADP _ 14 VMOD 24 about _ IN IN ADP _ 26 NMOD 25 a _ DT DT DET _ 24 AMOD 26 month _ NN NN NOUN _ 23 PMOD 27 after _ IN IN ADP _ 26 NMOD 28 the _ DT DT DET _ 29 NMOD 29 transaction _ NN NN NOUN _ 27 PMOD 30 . _ . . . _ 5 P 1 But _ CC CC CONJ _ 14 VMOD 2 about _ IN IN ADP _ 4 NMOD 3 25 _ CD CD NUM _ 2 AMOD 4 % _ NN NN NOUN _ 14 SUB 5 of _ IN IN ADP _ 4 NMOD 6 the _ DT DT DET _ 7 NMOD 7 insiders _ NN NNS NOUN _ 5 PMOD 8 , _ , , . _ 14 P 9 according _ VB VBG VERB _ 14 VMOD 10 to _ TO TO PRT _ 9 PMOD 11 SEC _ NN NNP NOUN _ 12 NMOD 12 figures _ NN NNS NOUN _ 10 PMOD 13 , _ , , . _ 14 P 14 file _ VB VBP VERB _ 0 ROOT 15 their _ PR PRP$ PRON _ 16 NMOD 16 reports _ NN NNS NOUN _ 14 OBJ 17 late _ RB RB ADV _ 14 VMOD 18 . _ . . . _ 14 P 1 The _ DT DT DET _ 2 NMOD 2 changes _ NN NNS NOUN _ 3 SUB 3 were _ VB VBD VERB _ 27 VMOD 4 proposed _ VB VBN VERB _ 3 VC 5 in _ IN IN ADP _ 4 VMOD 6 an _ DT DT DET _ 7 NMOD 7 effort _ NN NN NOUN _ 5 PMOD 8 to _ TO TO PRT _ 9 VMOD 9 streamline _ VB VB VERB _ 7 NMOD 10 federal _ JJ JJ ADJ _ 11 NMOD 11 bureaucracy _ NN NN NOUN _ 9 OBJ 12 and _ CC CC CONJ _ 9 VMOD 13 boost _ VB VB VERB _ 9 VMOD 14 compliance _ NN NN NOUN _ 13 OBJ 15 by _ IN IN ADP _ 14 NMOD 16 the _ DT DT DET _ 17 NMOD 17 executives _ NN NNS NOUN _ 15 PMOD 18 `` _ `` `` . _ 17 P 19 who _ WP WP PRON _ 17 NMOD 20 are _ VB VBP VERB _ 19 SBAR 21 really _ RB RB ADV _ 20 VMOD 22 calling _ VB VBG VERB _ 20 VC 23 the _ DT DT DET _ 24 NMOD 24 shots _ NN NNS NOUN _ 22 OBJ 25 , _ , , . _ 27 P 26 '' _ '' '' . _ 27 P 27 said _ VB VBD VERB _ 0 ROOT 28 Brian _ NN NNP NOUN _ 29 NMOD 29 Lane _ NN NNP NOUN _ 32 NMOD 30 , _ , , . _ 32 P 31 special _ JJ JJ ADJ _ 32 NMOD 32 counsel _ NN NN NOUN _ 27 SUB 33 at _ IN IN ADP _ 32 NMOD 34 the _ DT DT DET _ 36 NMOD 35 SEC _ NN NNP NOUN _ 36 NMOD 36 's _ PO POS PRT _ 37 NMOD 37 office _ NN NN NOUN _ 33 PMOD 38 of _ IN IN ADP _ 37 NMOD 39 disclosure _ NN NN NOUN _ 40 NMOD 40 policy _ NN NN NOUN _ 38 PMOD 41 , _ , , . _ 37 P 42 which _ WD WDT DET _ 37 NMOD 43 proposed _ VB VBD VERB _ 42 SBAR 44 the _ DT DT DET _ 45 NMOD 45 changes _ NN NNS NOUN _ 43 OBJ 46 . _ . . . _ 27 P 1 Investors _ NN NNS NOUN _ 7 NMOD 2 , _ , , . _ 7 P 3 money _ NN NN NOUN _ 4 NMOD 4 managers _ NN NNS NOUN _ 7 NMOD 5 and _ CC CC CONJ _ 7 NMOD 6 corporate _ JJ JJ ADJ _ 7 NMOD 7 officials _ NN NNS NOUN _ 8 SUB 8 had _ VB VBD VERB _ 20 VMOD 9 until _ IN IN ADP _ 8 VMOD 10 today _ NN NN NOUN _ 9 PMOD 11 to _ TO TO PRT _ 12 VMOD 12 comment _ VB VB VERB _ 8 VMOD 13 on _ IN IN ADP _ 12 VMOD 14 the _ DT DT DET _ 15 NMOD 15 proposals _ NN NNS NOUN _ 13 PMOD 16 , _ , , . _ 20 P 17 and _ CC CC CONJ _ 20 VMOD 18 the _ DT DT DET _ 19 NMOD 19 issue _ NN NN NOUN _ 20 SUB 20 has _ VB VBZ VERB _ 34 VMOD 21 produced _ VB VBN VERB _ 20 VC 22 more _ JJ JJR ADJ _ 23 NMOD 23 mail _ NN NN NOUN _ 21 OBJ 24 than _ IN IN ADP _ 23 NMOD 25 almost _ RB RB ADV _ 26 AMOD 26 any _ DT DT DET _ 28 NMOD 27 other _ JJ JJ ADJ _ 28 NMOD 28 issue _ NN NN NOUN _ 24 PMOD 29 in _ IN IN ADP _ 28 NMOD 30 memory _ NN NN NOUN _ 29 PMOD 31 , _ , , . _ 34 P 32 Mr. _ NN NNP NOUN _ 33 NMOD 33 Lane _ NN NNP NOUN _ 34 SUB 34 said _ VB VBD VERB _ 0 ROOT 35 . _ . . . _ 34 P 1 The _ DT DT DET _ 2 NMOD 2 SEC _ NN NNP NOUN _ 3 SUB 3 will _ MD MD VERB _ 14 VMOD 4 probably _ RB RB ADV _ 3 VMOD 5 vote _ VB VB VERB _ 3 VC 6 on _ IN IN ADP _ 5 VMOD 7 the _ DT DT DET _ 8 NMOD 8 proposal _ NN NN NOUN _ 6 PMOD 9 early _ RB RB ADV _ 11 NMOD 10 next _ JJ JJ ADJ _ 11 NMOD 11 year _ NN NN NOUN _ 5 VMOD 12 , _ , , . _ 14 P 13 he _ PR PRP PRON _ 14 SUB 14 said _ VB VBD VERB _ 0 ROOT 15 . _ . . . _ 14 P 1 Not _ RB RB ADV _ 6 SUB 2 all _ PD PDT DET _ 1 NMOD 3 those _ DT DT DET _ 1 NMOD 4 who _ WP WP PRON _ 1 NMOD 5 wrote _ VB VBD VERB _ 4 SBAR 6 oppose _ VB VBP VERB _ 0 ROOT 7 the _ DT DT DET _ 8 NMOD 8 changes _ NN NNS NOUN _ 6 OBJ 9 . _ . . . _ 6 P 1 The _ DT DT DET _ 2 NMOD 2 Committee _ NN NNP NOUN _ 13 SUB 3 on _ IN IN ADP _ 2 NMOD 4 Federal _ NN NNP NOUN _ 5 NMOD 5 Regulation _ NN NNP NOUN _ 3 PMOD 6 of _ IN IN ADP _ 5 NMOD 7 Securities _ NN NNPS NOUN _ 6 PMOD 8 for _ IN IN ADP _ 2 NMOD 9 the _ DT DT DET _ 12 NMOD 10 American _ NN NNP NOUN _ 12 NMOD 11 Bar _ NN NNP NOUN _ 12 NMOD 12 Association _ NN NNP NOUN _ 8 PMOD 13 argues _ VB VBZ VERB _ 0 ROOT 14 , _ , , . _ 13 P 15 for _ IN IN ADP _ 13 VMOD 16 example _ NN NN NOUN _ 15 PMOD 17 , _ , , . _ 13 P 18 in _ IN IN ADP _ 13 VMOD 19 its _ PR PRP$ PRON _ 21 NMOD 20 lengthy _ JJ JJ ADJ _ 21 NMOD 21 letter _ NN NN NOUN _ 18 PMOD 22 to _ TO TO PRT _ 21 NMOD 23 the _ DT DT DET _ 24 NMOD 24 SEC _ NN NNP NOUN _ 22 PMOD 25 , _ , , . _ 13 P 26 that _ IN IN ADP _ 13 VMOD 27 the _ DT DT DET _ 29 NMOD 28 proposed _ VB VBN VERB _ 29 NMOD 29 changes _ NN NNS NOUN _ 31 SUB 30 `` _ `` `` . _ 31 P 31 would _ MD MD VERB _ 26 SBAR 32 substantially _ RB RB ADV _ 33 VMOD 33 improve _ VB VB VERB _ 31 VC 34 the _ DT DT DET _ 36 NMOD 35 { _ -L -LRB- . _ 36 P 36 law _ NN NN NOUN _ 33 OBJ 37 } _ -R -RRB- . _ 36 P 38 by _ IN IN ADP _ 33 VMOD 39 conforming _ VB VBG VERB _ 38 PMOD 40 it _ PR PRP PRON _ 39 OBJ 41 more _ RB RBR ADV _ 42 AMOD 42 closely _ RB RB ADV _ 39 VMOD 43 to _ TO TO PRT _ 39 VMOD 44 contemporary _ JJ JJ ADJ _ 46 NMOD 45 business _ NN NN NOUN _ 46 NMOD 46 realities _ NN NNS NOUN _ 43 PMOD 47 . _ . . . _ 13 P 48 '' _ '' '' . _ 13 P 1 What _ WP WP PRON _ 12 VMOD 2 the _ DT DT DET _ 3 NMOD 3 investors _ NN NNS NOUN _ 9 SUB 4 who _ WP WP PRON _ 3 NMOD 5 oppose _ VB VBP VERB _ 4 SBAR 6 the _ DT DT DET _ 8 NMOD 7 proposed _ VB VBN VERB _ 8 NMOD 8 changes _ NN NNS NOUN _ 5 OBJ 9 object _ VB VBP VERB _ 1 SBAR 10 to _ TO TO PRT _ 9 VMOD 11 most _ RB RBS ADV _ 9 VMOD 12 is _ VB VBZ VERB _ 0 ROOT 13 the _ DT DT DET _ 14 NMOD 14 effect _ NN NN NOUN _ 12 PRD 15 they _ PR PRP PRON _ 16 SUB 16 say _ VB VBP VERB _ 14 NMOD 17 the _ DT DT DET _ 18 NMOD 18 proposal _ NN NN NOUN _ 19 SUB 19 would _ MD MD VERB _ 16 VMOD 20 have _ VB VB VERB _ 19 VC 21 on _ IN IN ADP _ 20 VMOD 22 their _ PR PRP$ PRON _ 23 NMOD 23 ability _ NN NN NOUN _ 21 PMOD 24 to _ TO TO PRT _ 25 VMOD 25 spot _ VB VB VERB _ 23 NMOD 26 telltale _ JJ JJ ADJ _ 28 NMOD 27 `` _ `` `` . _ 28 P 28 clusters _ NN NNS NOUN _ 36 NMOD 29 '' _ '' '' . _ 28 P 30 of _ IN IN ADP _ 28 NMOD 31 trading _ NN NN NOUN _ 32 NMOD 32 activity _ NN NN NOUN _ 30 PMOD 33 -- _ : : . _ 36 P 34 buying _ NN NN NOUN _ 36 NMOD 35 or _ CC CC CONJ _ 36 NMOD 36 selling _ NN NN NOUN _ 25 OBJ 37 by _ IN IN ADP _ 36 NMOD 38 more _ JJ JJR ADJ _ 39 AMOD 39 than _ IN IN ADP _ 43 NMOD 40 one _ CD CD NUM _ 39 AMOD 41 officer _ NN NN NOUN _ 43 NMOD 42 or _ CC CC CONJ _ 43 NMOD 43 director _ NN NN NOUN _ 37 PMOD 44 within _ IN IN ADP _ 36 NMOD 45 a _ DT DT DET _ 47 NMOD 46 short _ JJ JJ ADJ _ 47 NMOD 47 period _ NN NN NOUN _ 44 PMOD 48 of _ IN IN ADP _ 47 NMOD 49 time _ NN NN NOUN _ 48 PMOD 50 . _ . . . _ 12 P 1 According _ VB VBG VERB _ 9 VMOD 2 to _ TO TO PRT _ 1 PMOD 3 some _ DT DT DET _ 4 NMOD 4 estimates _ NN NNS NOUN _ 2 PMOD 5 , _ , , . _ 9 P 6 the _ DT DT DET _ 8 NMOD 7 rule _ NN NN NOUN _ 8 NMOD 8 changes _ NN NNS NOUN _ 9 SUB 9 would _ MD MD VERB _ 0 ROOT 10 cut _ VB VB VERB _ 9 VC 11 insider _ NN NN NOUN _ 12 NMOD 12 filings _ NN NNS NOUN _ 10 OBJ 13 by _ IN IN ADP _ 10 VMOD 14 more _ JJ JJR ADJ _ 13 PMOD 15 than _ IN IN ADP _ 14 NMOD 16 a _ DT DT DET _ 17 NMOD 17 third _ NN NN NOUN _ 15 PMOD 18 . _ . . . _ 9 P 1 The _ DT DT DET _ 3 NMOD 2 SEC _ NN NNP NOUN _ 3 NMOD 3 's _ PO POS PRT _ 5 NMOD 4 Mr. _ NN NNP NOUN _ 5 NMOD 5 Lane _ NN NNP NOUN _ 7 SUB 6 vehemently _ RB RB ADV _ 7 VMOD 7 disputed _ VB VBD VERB _ 0 ROOT 8 those _ DT DT DET _ 9 NMOD 9 estimates _ NN NNS NOUN _ 7 OBJ 10 . _ . . . _ 7 P 1 The _ DT DT DET _ 2 NMOD 2 rules _ NN NNS NOUN _ 3 SUB 3 will _ MD MD VERB _ 23 VMOD 4 eliminate _ VB VB VERB _ 3 VC 5 filings _ NN NNS NOUN _ 7 NMOD 6 policy-making _ VB VBG VERB _ 7 NMOD 7 divisions _ NN NNS NOUN _ 4 OBJ 8 , _ , , . _ 7 P 9 such _ JJ JJ ADJ _ 10 PMOD 10 as _ IN IN ADP _ 7 NMOD 11 sales _ NN NNS NOUN _ 19 NMOD 12 , _ , , . _ 19 P 13 marketing _ NN NN NOUN _ 19 NMOD 14 , _ , , . _ 19 P 15 finance _ NN NN NOUN _ 19 NMOD 16 and _ CC CC CONJ _ 19 NMOD 17 research _ NN NN NOUN _ 19 NMOD 18 and _ CC CC CONJ _ 19 NMOD 19 development _ NN NN NOUN _ 10 PMOD 20 , _ , , . _ 23 P 21 Mr. _ NN NNP NOUN _ 22 NMOD 22 Lane _ NN NNP NOUN _ 23 SUB 23 said _ VB VBD VERB _ 0 ROOT 24 . _ . . . _ 23 P 1 The _ DT DT DET _ 3 NMOD 2 proposed _ VB VBN VERB _ 3 NMOD 3 rules _ NN NNS NOUN _ 5 SUB 4 also _ RB RB ADV _ 5 VMOD 5 would _ MD MD VERB _ 18 VMOD 6 be _ VB VB VERB _ 5 VC 7 tougher _ JJ JJR ADJ _ 6 PRD 8 on _ IN IN ADP _ 7 AMOD 9 the _ DT DT DET _ 10 NMOD 10 insiders _ NN NNS NOUN _ 8 PMOD 11 still _ RB RB ADV _ 10 NMOD 12 required _ JJ JJ ADJ _ 11 VMOD 13 to _ TO TO PRT _ 14 VMOD 14 file _ VB VB VERB _ 11 VMOD 15 reports _ NN NNS NOUN _ 14 OBJ 16 , _ , , . _ 18 P 17 he _ PR PRP PRON _ 18 SUB 18 said _ VB VBD VERB _ 0 ROOT 19 . _ . . . _ 18 P 1 Companies _ NN NNS NOUN _ 2 SUB 2 would _ MD MD VERB _ 0 ROOT 3 be _ VB VB VERB _ 2 VC 4 compelled _ VB VBN VERB _ 3 VC 5 to _ TO TO PRT _ 6 VMOD 6 publish _ VB VB VERB _ 4 VMOD 7 in _ IN IN ADP _ 6 VMOD 8 annual _ JJ JJ ADJ _ 10 NMOD 9 proxy _ NN NN NOUN _ 10 NMOD 10 statements _ NN NNS NOUN _ 7 PMOD 11 the _ DT DT DET _ 12 NMOD 12 names _ NN NNS NOUN _ 6 OBJ 13 of _ IN IN ADP _ 12 NMOD 14 insiders _ NN NNS NOUN _ 13 PMOD 15 who _ WP WP PRON _ 14 NMOD 16 fail _ VB VBP VERB _ 15 SBAR 17 to _ TO TO PRT _ 18 VMOD 18 file _ VB VB VERB _ 16 VMOD 19 reports _ NN NNS NOUN _ 18 OBJ 20 on _ IN IN ADP _ 18 VMOD 21 time _ NN NN NOUN _ 20 PMOD 22 . _ . . . _ 2 P 1 Considered _ VB VBN VERB _ 18 VMOD 2 as _ IN IN ADP _ 1 VMOD 3 a _ DT DT DET _ 4 NMOD 4 whole _ NN NN NOUN _ 2 PMOD 5 , _ , , . _ 8 P 6 Mr. _ NN NNP NOUN _ 7 NMOD 7 Lane _ NN NNP NOUN _ 8 SUB 8 said _ VB VBD VERB _ 18 VMOD 9 , _ , , . _ 8 P 10 the _ DT DT DET _ 11 NMOD 11 filings _ NN NNS NOUN _ 18 SUB 12 required _ VB VBN VERB _ 11 NMOD 13 under _ IN IN ADP _ 12 VMOD 14 the _ DT DT DET _ 16 NMOD 15 proposed _ VB VBN VERB _ 16 NMOD 16 rules _ NN NNS NOUN _ 13 PMOD 17 `` _ `` `` . _ 18 P 18 will _ MD MD VERB _ 0 ROOT 19 be _ VB VB VERB _ 18 VC 20 at _ IN IN ADP _ 21 AMOD 21 least _ RB RBS ADV _ 19 PRD 22 as _ IN IN ADP _ 21 AMOD 23 effective _ JJ JJ ADJ _ 21 AMOD 24 , _ , , . _ 25 P 25 if _ IN IN ADP _ 21 AMOD 26 not _ RB RB ADV _ 25 DEP 27 more _ RB RBR ADV _ 28 AMOD 28 so _ RB RB ADV _ 26 DEP 29 , _ , , . _ 25 P 30 for _ IN IN ADP _ 19 VMOD 31 investors _ NN NNS NOUN _ 30 PMOD 32 following _ VB VBG VERB _ 31 NMOD 33 transactions _ NN NNS NOUN _ 32 OBJ 34 . _ . . . _ 18 P 35 '' _ '' '' . _ 18 P 1 But _ CC CC CONJ _ 24 VMOD 2 Robert _ NN NNP NOUN _ 3 NMOD 3 Gabele _ NN NNP NOUN _ 5 NMOD 4 , _ , , . _ 5 P 5 president _ NN NN NOUN _ 24 SUB 6 of _ IN IN ADP _ 5 NMOD 7 Invest\/Net _ NN NNP NOUN _ 15 NMOD 8 , _ , , . _ 15 P 9 a _ DT DT DET _ 15 NMOD 10 North _ NN NNP NOUN _ 13 NMOD 11 Miami _ NN NNP NOUN _ 13 NMOD 12 , _ , , . _ 13 P 13 Fla. _ NN NNP NOUN _ 15 NMOD 14 , _ , , . _ 13 P 15 company _ NN NN NOUN _ 6 PMOD 16 that _ WD WDT DET _ 15 NMOD 17 packages _ VB VBZ VERB _ 16 SBAR 18 and _ CC CC CONJ _ 17 VMOD 19 sells _ VB VBZ VERB _ 17 VMOD 20 the _ DT DT DET _ 22 NMOD 21 insider-trading _ NN NN NOUN _ 22 NMOD 22 data _ NN NNS NOUN _ 17 OBJ 23 , _ , , . _ 5 P 24 said _ VB VBD VERB _ 0 ROOT 25 the _ DT DT DET _ 26 NMOD 26 proposal _ NN NN NOUN _ 27 SUB 27 is _ VB VBZ VERB _ 24 VMOD 28 worded _ VB VBN VERB _ 27 VC 29 so _ RB RB ADV _ 28 VMOD 30 vaguely _ RB RB ADV _ 29 AMOD 31 that _ IN IN ADP _ 29 AMOD 32 key _ JJ JJ ADJ _ 33 NMOD 33 officials _ NN NNS NOUN _ 34 SUB 34 may _ MD MD VERB _ 31 SBAR 35 fail _ VB VB VERB _ 34 VC 36 to _ TO TO PRT _ 37 VMOD 37 file _ VB VB VERB _ 35 VMOD 38 the _ DT DT DET _ 39 NMOD 39 reports _ NN NNS NOUN _ 37 OBJ 40 . _ . . . _ 24 P 1 Many _ JJ JJ ADJ _ 2 NMOD 2 investors _ NN NNS NOUN _ 3 SUB 3 wrote _ VB VBD VERB _ 0 ROOT 4 asking _ VB VBG VERB _ 3 VMOD 5 the _ DT DT DET _ 6 NMOD 6 SEC _ NN NNP NOUN _ 4 OBJ 7 to _ TO TO PRT _ 8 VMOD 8 require _ VB VB VERB _ 4 VMOD 9 insiders _ NN NNS NOUN _ 11 SUB 10 to _ TO TO PRT _ 11 VMOD 11 report _ VB VB VERB _ 8 VMOD 12 their _ PR PRP$ PRON _ 15 NMOD 13 purchases _ NN NNS NOUN _ 15 NMOD 14 and _ CC CC CONJ _ 15 NMOD 15 sales _ NN NNS NOUN _ 11 OBJ 16 immediately _ RB RB ADV _ 18 AMOD 17 , _ , , . _ 18 P 18 not _ RB RB ADV _ 11 VMOD 19 a _ DT DT DET _ 20 NMOD 20 month _ NN NN NOUN _ 21 AMOD 21 later _ RB RB ADV _ 18 AMOD 22 . _ . . . _ 3 P 1 But _ CC CC CONJ _ 4 VMOD 2 Mr. _ NN NNP NOUN _ 3 NMOD 3 Lane _ NN NNP NOUN _ 4 SUB 4 said _ VB VBD VERB _ 0 ROOT 5 that _ IN IN ADP _ 4 VMOD 6 while _ IN IN ADP _ 15 VMOD 7 the _ DT DT DET _ 8 NMOD 8 SEC _ NN NNP NOUN _ 9 SUB 9 regulates _ VB VBZ VERB _ 6 SBAR 10 who _ WP WP PRON _ 9 VMOD 11 files _ VB VBZ VERB _ 10 SBAR 12 , _ , , . _ 15 P 13 the _ DT DT DET _ 14 NMOD 14 law _ NN NN NOUN _ 15 SUB 15 tells _ VB VBZ VERB _ 5 SBAR 16 them _ PR PRP PRON _ 15 OBJ 17 when _ WR WRB ADV _ 15 VMOD 18 to _ TO TO PRT _ 19 VMOD 19 do _ VB VB VERB _ 17 SBAR 20 so _ RB RB ADV _ 19 PRD 21 . _ . . . _ 4 P 1 Investors _ NN NNS NOUN _ 9 SUB 2 who _ WP WP PRON _ 1 NMOD 3 want _ VB VBP VERB _ 2 SBAR 4 to _ TO TO PRT _ 5 VMOD 5 change _ VB VB VERB _ 3 VMOD 6 the _ DT DT DET _ 8 NMOD 7 required _ JJ JJ ADJ _ 8 NMOD 8 timing _ NN NN NOUN _ 5 OBJ 9 should _ MD MD VERB _ 17 VMOD 10 write _ VB VB VERB _ 9 VC 11 their _ PR PRP$ PRON _ 12 NMOD 12 representatives _ NN NNS NOUN _ 10 OBJ 13 in _ IN IN ADP _ 12 NMOD 14 Congress _ NN NNP NOUN _ 13 PMOD 15 , _ , , . _ 17 P 16 he _ PR PRP PRON _ 17 SUB 17 added _ VB VBD VERB _ 0 ROOT 18 . _ . . . _ 17 P 1 The _ DT DT DET _ 2 NMOD 2 SEC _ NN NNP NOUN _ 3 SUB 3 would _ MD MD VERB _ 22 VMOD 4 likely _ RB RB ADV _ 3 VMOD 5 be _ VB VB VERB _ 3 VC 6 amenable _ JJ JJ ADJ _ 5 PRD 7 to _ TO TO PRT _ 6 AMOD 8 legislation _ NN NN NOUN _ 7 PMOD 9 that _ WD WDT DET _ 8 NMOD 10 required _ VB VBD VERB _ 9 SBAR 11 insiders _ NN NNS NOUN _ 13 SUB 12 to _ TO TO PRT _ 13 VMOD 13 file _ VB VB VERB _ 10 VMOD 14 transactions _ NN NNS NOUN _ 13 OBJ 15 on _ IN IN ADP _ 13 VMOD 16 a _ DT DT DET _ 19 NMOD 17 more _ RB RBR ADV _ 18 AMOD 18 timely _ JJ JJ ADJ _ 19 NMOD 19 basis _ NN NN NOUN _ 15 PMOD 20 , _ , , . _ 22 P 21 he _ PR PRP PRON _ 22 SUB 22 said _ VB VBD VERB _ 0 ROOT 23 . _ . . . _ 22 P 1 The _ DT DT DET _ 3 NMOD 2 nation _ NN NN NOUN _ 3 NMOD 3 's _ PO POS PRT _ 6 NMOD 4 largest _ JJ JJS ADJ _ 6 NMOD 5 pension _ NN NN NOUN _ 6 NMOD 6 fund _ NN NN NOUN _ 17 SUB 7 , _ , , . _ 6 P 8 which _ WD WDT DET _ 6 NMOD 9 oversees _ VB VBZ VERB _ 8 SBAR 10 $ _ $ $ . _ 9 P 11 80 _ CD CD NUM _ 10 AMOD 12 billion _ CD CD NUM _ 10 AMOD 13 for _ IN IN ADP _ 9 VMOD 14 college _ NN NN NOUN _ 15 NMOD 15 employees _ NN NNS NOUN _ 13 PMOD 16 , _ , , . _ 6 P 17 plans _ VB VBZ VERB _ 0 ROOT 18 to _ TO TO PRT _ 19 VMOD 19 offer _ VB VB VERB _ 17 VMOD 20 two _ CD CD NUM _ 23 NMOD 21 new _ JJ JJ ADJ _ 23 NMOD 22 investment _ NN NN NOUN _ 23 NMOD 23 options _ NN NNS NOUN _ 19 OBJ 24 to _ TO TO PRT _ 19 VMOD 25 its _ PR PRP$ PRON _ 28 NMOD 26 1.2 _ CD CD NUM _ 27 AMOD 27 million _ CD CD NUM _ 28 NMOD 28 participants _ NN NNS NOUN _ 24 PMOD 29 . _ . . . _ 17 P 1 The _ DT DT DET _ 9 NMOD 2 Teachers _ NN NNPS NOUN _ 9 NMOD 3 Insurance _ NN NNP NOUN _ 9 NMOD 4 and _ CC CC CONJ _ 9 NMOD 5 Annuity _ NN NNP NOUN _ 9 NMOD 6 Association-College _ NN NNP NOUN _ 9 NMOD 7 Retirement _ NN NNP NOUN _ 9 NMOD 8 Equities _ NN NNPS NOUN _ 9 NMOD 9 Fund _ NN NNP NOUN _ 10 SUB 10 said _ VB VBD VERB _ 0 ROOT 11 it _ PR PRP PRON _ 12 SUB 12 will _ MD MD VERB _ 10 VMOD 13 introduce _ VB VB VERB _ 12 VC 14 a _ DT DT DET _ 18 NMOD 15 stock _ NN NN NOUN _ 18 NMOD 16 and _ CC CC CONJ _ 18 NMOD 17 bond _ NN NN NOUN _ 18 NMOD 18 fund _ NN NN NOUN _ 32 NMOD 19 that _ WD WDT DET _ 18 NMOD 20 will _ MD MD VERB _ 19 SBAR 21 invest _ VB VB VERB _ 20 VC 22 in _ IN IN ADP _ 21 VMOD 23 `` _ `` `` . _ 27 P 24 socially _ RB RB ADV _ 25 AMOD 25 responsible _ JJ JJ ADJ _ 27 NMOD 26 '' _ '' '' . _ 27 P 27 companies _ NN NNS NOUN _ 22 PMOD 28 , _ , , . _ 32 P 29 and _ CC CC CONJ _ 32 NMOD 30 a _ DT DT DET _ 32 NMOD 31 bond _ NN NN NOUN _ 32 NMOD 32 fund _ NN NN NOUN _ 13 OBJ 33 . _ . . . _ 10 P 1 Both _ DT DT DET _ 2 NMOD 2 funds _ NN NNS NOUN _ 3 SUB 3 are _ VB VBP VERB _ 0 ROOT 4 expected _ VB VBN VERB _ 3 VC 5 to _ TO TO PRT _ 6 VMOD 6 begin _ VB VB VERB _ 4 VMOD 7 operation _ NN NN NOUN _ 6 OBJ 8 around _ IN IN ADP _ 6 VMOD 9 March _ NN NNP NOUN _ 8 PMOD 10 1 _ CD CD NUM _ 9 NMOD 11 , _ , , . _ 6 P 12 subject _ JJ JJ ADJ _ 6 VMOD 13 to _ TO TO PRT _ 12 AMOD 14 Securities _ NN NNPS NOUN _ 18 NMOD 15 and _ CC CC CONJ _ 18 NMOD 16 Exchange _ NN NNP NOUN _ 18 NMOD 17 Commission _ NN NNP NOUN _ 18 NMOD 18 approval _ NN NN NOUN _ 13 PMOD 19 . _ . . . _ 3 P 1 For _ IN IN ADP _ 14 VMOD 2 its _ PR PRP$ PRON _ 3 NMOD 3 employees _ NN NNS NOUN _ 5 SUB 4 to _ TO TO PRT _ 5 VMOD 5 sign _ VB VB VERB _ 1 SBAR 6 up _ RP RP PRT _ 5 VMOD 7 for _ IN IN ADP _ 5 VMOD 8 the _ DT DT DET _ 9 NMOD 9 options _ NN NNS NOUN _ 7 PMOD 10 , _ , , . _ 14 P 11 a _ DT DT DET _ 12 NMOD 12 college _ NN NN NOUN _ 14 SUB 13 also _ RB RB ADV _ 14 VMOD 14 must _ MD MD VERB _ 0 ROOT 15 approve _ VB VB VERB _ 14 VC 16 the _ DT DT DET _ 17 NMOD 17 plan _ NN NN NOUN _ 15 OBJ 18 . _ . . . _ 14 P 1 Some _ DT DT DET _ 3 NMOD 2 4,300 _ CD CD NUM _ 1 AMOD 3 institutions _ NN NNS NOUN _ 4 SUB 4 are _ VB VBP VERB _ 0 ROOT 5 part _ NN NN NOUN _ 4 PRD 6 of _ IN IN ADP _ 5 NMOD 7 the _ DT DT DET _ 9 NMOD 8 pension _ NN NN NOUN _ 9 NMOD 9 fund _ NN NN NOUN _ 6 PMOD 10 . _ . . . _ 4 P 1 The _ DT DT DET _ 3 NMOD 2 new _ JJ JJ ADJ _ 3 NMOD 3 options _ NN NNS NOUN _ 4 SUB 4 carry _ VB VBP VERB _ 0 ROOT 5 out _ RP RP PRT _ 4 VMOD 6 part _ NN NN NOUN _ 4 OBJ 7 of _ IN IN ADP _ 6 NMOD 8 an _ DT DT DET _ 9 NMOD 9 agreement _ NN NN NOUN _ 7 PMOD 10 that _ IN IN ADP _ 9 NMOD 11 the _ DT DT DET _ 13 NMOD 12 pension _ NN NN NOUN _ 13 NMOD 13 fund _ NN NN NOUN _ 30 SUB 14 , _ , , . _ 30 P 15 under _ IN IN ADP _ 30 VMOD 16 pressure _ NN NN NOUN _ 15 PMOD 17 to _ TO TO PRT _ 18 VMOD 18 relax _ VB VB VERB _ 16 NMOD 19 its _ PR PRP$ PRON _ 22 NMOD 20 strict _ JJ JJ ADJ _ 22 NMOD 21 participation _ NN NN NOUN _ 22 NMOD 22 rules _ NN NNS NOUN _ 18 OBJ 23 and _ CC CC CONJ _ 18 VMOD 24 to _ TO TO PRT _ 25 VMOD 25 provide _ VB VB VERB _ 18 VMOD 26 more _ JJ JJR ADJ _ 28 NMOD 27 investment _ NN NN NOUN _ 28 NMOD 28 options _ NN NNS NOUN _ 25 OBJ 29 , _ , , . _ 30 P 30 reached _ VB VBN VERB _ 10 SBAR 31 with _ IN IN ADP _ 30 VMOD 32 the _ DT DT DET _ 33 NMOD 33 SEC _ NN NNP NOUN _ 31 PMOD 34 in _ IN IN ADP _ 30 VMOD 35 December _ NN NNP NOUN _ 34 PMOD 36 . _ . . . _ 4 P 1 The _ DT DT DET _ 7 NMOD 2 new _ JJ JJ ADJ _ 7 NMOD 3 `` _ `` `` . _ 7 P 4 social _ JJ JJ ADJ _ 7 NMOD 5 choice _ NN NN NOUN _ 7 NMOD 6 '' _ '' '' . _ 7 P 7 fund _ NN NN NOUN _ 8 SUB 8 will _ MD MD VERB _ 0 ROOT 9 shun _ VB VB VERB _ 8 VC 10 securities _ NN NNS NOUN _ 9 OBJ 11 of _ IN IN ADP _ 10 NMOD 12 companies _ NN NNS NOUN _ 11 PMOD 13 linked _ VB VBN VERB _ 12 NMOD 14 to _ TO TO PRT _ 13 VMOD 15 South _ NN NNP NOUN _ 16 NMOD 16 Africa _ NN NNP NOUN _ 26 NMOD 17 , _ , , . _ 26 P 18 nuclear _ JJ JJ ADJ _ 19 NMOD 19 power _ NN NN NOUN _ 26 NMOD 20 and _ CC CC CONJ _ 26 NMOD 21 in _ IN IN ADP _ 26 NMOD 22 some _ DT DT DET _ 23 NMOD 23 cases _ NN NNS NOUN _ 21 PMOD 24 , _ , , . _ 26 P 25 Northern _ NN NNP NOUN _ 26 NMOD 26 Ireland _ NN NNP NOUN _ 14 PMOD 27 . _ . . . _ 8 P 1 Also _ RB RB ADV _ 3 VMOD 2 excluded _ VB VBN VERB _ 3 VMOD 3 will _ MD MD VERB _ 0 ROOT 4 be _ VB VB VERB _ 3 VC 5 investments _ NN NNS NOUN _ 3 SUB 6 in _ IN IN ADP _ 5 NMOD 7 companies _ NN NNS NOUN _ 6 PMOD 8 with _ IN IN ADP _ 7 NMOD 9 `` _ `` `` . _ 12 P 10 significant _ JJ JJ ADJ _ 12 NMOD 11 '' _ '' '' . _ 12 P 12 business _ NN NN NOUN _ 8 PMOD 13 stemming _ VB VBG VERB _ 12 NMOD 14 from _ IN IN ADP _ 13 VMOD 15 weapons _ NN NNS NOUN _ 16 NMOD 16 manufacture _ NN NN NOUN _ 21 NMOD 17 , _ , , . _ 21 P 18 alcoholic _ JJ JJ ADJ _ 19 NMOD 19 beverages _ NN NNS NOUN _ 21 NMOD 20 or _ CC CC CONJ _ 21 NMOD 21 tobacco _ NN NN NOUN _ 14 PMOD 22 . _ . . . _ 3 P 1 Sixty _ CD CD NUM _ 2 NMOD 2 percent _ NN NN NOUN _ 6 SUB 3 of _ IN IN ADP _ 2 NMOD 4 the _ DT DT DET _ 5 NMOD 5 fund _ NN NN NOUN _ 3 PMOD 6 will _ MD MD VERB _ 0 ROOT 7 be _ VB VB VERB _ 6 VC 8 invested _ VB VBN VERB _ 7 VC 9 in _ IN IN ADP _ 8 VMOD 10 stocks _ NN NNS NOUN _ 9 PMOD 11 , _ , , . _ 8 P 12 with _ IN IN ADP _ 8 VMOD 13 the _ DT DT DET _ 14 NMOD 14 rest _ NN NN NOUN _ 15 SUB 15 going _ VB VBG VERB _ 12 PMOD 16 into _ IN IN ADP _ 15 VMOD 17 bonds _ NN NNS NOUN _ 20 NMOD 18 or _ CC CC CONJ _ 20 NMOD 19 short-term _ JJ JJ ADJ _ 20 NMOD 20 investments _ NN NNS NOUN _ 16 PMOD 21 . _ . . . _ 6 P 1 The _ DT DT DET _ 3 NMOD 2 bond _ NN NN NOUN _ 3 NMOD 3 fund _ NN NN NOUN _ 4 SUB 4 will _ MD MD VERB _ 0 ROOT 5 invest _ VB VB VERB _ 4 VC 6 in _ IN IN ADP _ 5 VMOD 7 high-grade _ JJ JJ ADJ _ 9 AMOD 8 or _ CC CC CONJ _ 9 AMOD 9 medium-grade _ JJ JJ ADJ _ 10 NMOD 10 bonds _ NN NNS NOUN _ 15 NMOD 11 , _ , , . _ 15 P 12 mortgages _ NN NNS NOUN _ 15 NMOD 13 or _ CC CC CONJ _ 15 NMOD 14 asset-backed _ JJ JJ ADJ _ 15 NMOD 15 securities _ NN NNS NOUN _ 6 PMOD 16 , _ , , . _ 15 P 17 including _ VB VBG VERB _ 15 NMOD 18 as _ JJ JJ ADJ _ 20 AMOD 19 much _ RB RB ADV _ 20 AMOD 20 as _ IN IN ADP _ 22 NMOD 21 15 _ CD CD NUM _ 20 AMOD 22 % _ NN NN NOUN _ 17 PMOD 23 in _ IN IN ADP _ 22 NMOD 24 foreign _ JJ JJ ADJ _ 25 NMOD 25 securities _ NN NNS NOUN _ 23 PMOD 26 . _ . . . _ 4 P 1 The _ DT DT DET _ 2 NMOD 2 fund _ NN NN NOUN _ 4 SUB 3 also _ RB RB ADV _ 4 VMOD 4 might _ MD MD VERB _ 0 ROOT 5 buy _ VB VB VERB _ 4 VC 6 and _ CC CC CONJ _ 5 VMOD 7 sell _ VB VB VERB _ 5 VMOD 8 futures _ NN NNS NOUN _ 11 NMOD 9 and _ CC CC CONJ _ 11 NMOD 10 options _ NN NNS NOUN _ 11 NMOD 11 contracts _ NN NNS NOUN _ 5 OBJ 12 , _ , , . _ 5 P 13 subject _ JJ JJ ADJ _ 5 VMOD 14 to _ TO TO PRT _ 13 AMOD 15 approval _ NN NN NOUN _ 14 PMOD 16 by _ IN IN ADP _ 15 NMOD 17 the _ DT DT DET _ 22 NMOD 18 New _ NN NNP NOUN _ 22 NMOD 19 York _ NN NNP NOUN _ 22 NMOD 20 State _ NN NNP NOUN _ 22 NMOD 21 Insurance _ NN NNP NOUN _ 22 NMOD 22 Department _ NN NNP NOUN _ 16 PMOD 23 . _ . . . _ 4 P 1 Under _ IN IN ADP _ 7 VMOD 2 two _ CD CD NUM _ 4 NMOD 3 new _ JJ JJ ADJ _ 4 NMOD 4 features _ NN NNS NOUN _ 1 PMOD 5 , _ , , . _ 7 P 6 participants _ NN NNS NOUN _ 7 SUB 7 will _ MD MD VERB _ 0 ROOT 8 be _ VB VB VERB _ 7 VC 9 able _ JJ JJ ADJ _ 8 VMOD 10 to _ TO TO PRT _ 11 VMOD 11 transfer _ VB VB VERB _ 9 AMOD 12 money _ NN NN NOUN _ 11 OBJ 13 from _ IN IN ADP _ 11 VMOD 14 the _ DT DT DET _ 16 NMOD 15 new _ JJ JJ ADJ _ 16 NMOD 16 funds _ NN NNS NOUN _ 13 PMOD 17 to _ TO TO PRT _ 11 VMOD 18 other _ JJ JJ ADJ _ 20 NMOD 19 investment _ NN NN NOUN _ 20 NMOD 20 funds _ NN NNS NOUN _ 17 PMOD 21 or _ CC CC CONJ _ 11 VMOD 22 , _ , , . _ 23 P 23 if _ IN IN ADP _ 29 VMOD 24 their _ PR PRP$ PRON _ 25 NMOD 25 jobs _ NN NNS NOUN _ 26 SUB 26 are _ VB VBP VERB _ 23 SBAR 27 terminated _ VB VBN VERB _ 26 VC 28 , _ , , . _ 23 P 29 receive _ VB VB VERB _ 11 VMOD 30 cash _ NN NN NOUN _ 29 OBJ 31 from _ IN IN ADP _ 29 VMOD 32 the _ DT DT DET _ 33 NMOD 33 funds _ NN NNS NOUN _ 31 PMOD 34 . _ . . . _ 7 P 1 The _ DT DT DET _ 3 NMOD 2 investment _ NN NN NOUN _ 3 NMOD 3 choices _ NN NNS NOUN _ 10 SUB 4 offered _ VB VBN VERB _ 3 NMOD 5 by _ IN IN ADP _ 4 VMOD 6 the _ DT DT DET _ 8 NMOD 7 pension _ NN NN NOUN _ 8 NMOD 8 fund _ NN NN NOUN _ 5 PMOD 9 currently _ RB RB ADV _ 10 VMOD 10 are _ VB VBP VERB _ 0 ROOT 11 limited _ VB VBN VERB _ 10 VC 12 to _ TO TO PRT _ 11 VMOD 13 a _ DT DT DET _ 15 NMOD 14 stock _ NN NN NOUN _ 15 NMOD 15 fund _ NN NN NOUN _ 22 NMOD 16 , _ , , . _ 22 P 17 an _ DT DT DET _ 18 NMOD 18 annuity _ NN NN NOUN _ 22 NMOD 19 and _ CC CC CONJ _ 22 NMOD 20 a _ DT DT DET _ 22 NMOD 21 money-market _ NN NN NOUN _ 22 NMOD 22 fund _ NN NN NOUN _ 12 PMOD 23 . _ . . . _ 10 P 1 New _ NN NNP NOUN _ 4 NMOD 2 Brunswick _ NN NNP NOUN _ 4 NMOD 3 Scientific _ NN NNP NOUN _ 4 NMOD 4 Co. _ NN NNP NOUN _ 7 NMOD 5 , _ , , . _ 7 P 6 a _ DT DT DET _ 7 NMOD 7 maker _ NN NN NOUN _ 14 SUB 8 of _ IN IN ADP _ 7 NMOD 9 biotechnology _ NN NN NOUN _ 12 NMOD 10 instrumentation _ NN NN NOUN _ 12 NMOD 11 and _ CC CC CONJ _ 12 NMOD 12 equipment _ NN NN NOUN _ 8 PMOD 13 , _ , , . _ 7 P 14 said _ VB VBD VERB _ 0 ROOT 15 it _ PR PRP PRON _ 16 SUB 16 adopted _ VB VBD VERB _ 14 VMOD 17 an _ DT DT DET _ 19 NMOD 18 anti-takeover _ JJ JJ ADJ _ 19 NMOD 19 plan _ NN NN NOUN _ 16 OBJ 20 giving _ VB VBG VERB _ 19 NMOD 21 shareholders _ NN NNS NOUN _ 20 OBJ 22 the _ DT DT DET _ 23 NMOD 23 right _ NN NN NOUN _ 20 OBJ 24 to _ TO TO PRT _ 25 VMOD 25 purchase _ VB VB VERB _ 23 NMOD 26 shares _ NN NNS NOUN _ 25 OBJ 27 at _ IN IN ADP _ 25 VMOD 28 half _ JJ JJ ADJ _ 29 NMOD 29 price _ NN NN NOUN _ 27 PMOD 30 under _ IN IN ADP _ 25 VMOD 31 certain _ JJ JJ ADJ _ 32 NMOD 32 conditions _ NN NNS NOUN _ 30 PMOD 33 . _ . . . _ 14 P 1 The _ DT DT DET _ 2 NMOD 2 company _ NN NN NOUN _ 3 SUB 3 said _ VB VBD VERB _ 0 ROOT 4 the _ DT DT DET _ 5 NMOD 5 plan _ NN NN NOUN _ 13 SUB 6 , _ , , . _ 13 P 7 under _ IN IN ADP _ 9 PRD 8 review _ NN NN NOUN _ 7 PMOD 9 for _ IN IN ADP _ 13 VMOD 10 some _ DT DT DET _ 11 NMOD 11 time _ NN NN NOUN _ 9 PMOD 12 , _ , , . _ 13 P 13 will _ MD MD VERB _ 3 VMOD 14 protect _ VB VB VERB _ 13 VC 15 shareholders _ NN NNS NOUN _ 14 OBJ 16 against _ IN IN ADP _ 14 VMOD 17 `` _ `` `` . _ 16 P 18 abusive _ JJ JJ ADJ _ 20 NMOD 19 takeover _ NN NN NOUN _ 20 NMOD 20 tactics _ NN NNS NOUN _ 16 PMOD 21 . _ . . . _ 3 P 1 W. _ NN NNP NOUN _ 3 NMOD 2 Ed _ NN NNP NOUN _ 3 NMOD 3 Tyler _ NN NNP NOUN _ 12 NMOD 4 , _ , , . _ 12 P 5 37 _ CD CD NUM _ 6 NMOD 6 years _ NN NNS NOUN _ 7 AMOD 7 old _ JJ JJ ADJ _ 12 NMOD 8 , _ , , . _ 12 P 9 a _ DT DT DET _ 12 NMOD 10 senior _ JJ JJ ADJ _ 12 NMOD 11 vice _ NN NN NOUN _ 12 NMOD 12 president _ NN NN NOUN _ 18 SUB 13 at _ IN IN ADP _ 12 NMOD 14 this _ DT DT DET _ 16 NMOD 15 printing _ VB VBG VERB _ 16 NMOD 16 concern _ NN NN NOUN _ 13 PMOD 17 , _ , , . _ 12 P 18 was _ VB VBD VERB _ 0 ROOT 19 elected _ VB VBN VERB _ 18 VC 20 president _ NN NN NOUN _ 28 NMOD 21 of _ IN IN ADP _ 20 NMOD 22 its _ PR PRP$ PRON _ 24 NMOD 23 technology _ NN NN NOUN _ 24 NMOD 24 group _ NN NN NOUN _ 21 PMOD 25 , _ , , . _ 28 P 26 a _ DT DT DET _ 28 NMOD 27 new _ JJ JJ ADJ _ 28 NMOD 28 position _ NN NN NOUN _ 19 VMOD 29 . _ . . . _ 18 P 1 Solo _ JJ JJ ADJ _ 3 NMOD 2 woodwind _ NN NN NOUN _ 3 NMOD 3 players _ NN NNS NOUN _ 4 SUB 4 have _ VB VBP VERB _ 0 ROOT 5 to _ TO TO PRT _ 6 VMOD 6 be _ VB VB VERB _ 4 VMOD 7 creative _ JJ JJ ADJ _ 6 PRD 8 if _ IN IN ADP _ 4 VMOD 9 they _ PR PRP PRON _ 10 SUB 10 want _ VB VBP VERB _ 8 SBAR 11 to _ TO TO PRT _ 12 VMOD 12 work _ VB VB VERB _ 10 VMOD 13 a _ DT DT DET _ 14 NMOD 14 lot _ NN NN NOUN _ 12 VMOD 15 , _ , , . _ 4 P 16 because _ IN IN ADP _ 4 VMOD 17 their _ PR PRP$ PRON _ 21 NMOD 18 repertoire _ NN NN NOUN _ 21 NMOD 19 and _ CC CC CONJ _ 21 NMOD 20 audience _ NN NN NOUN _ 21 NMOD 21 appeal _ NN NN NOUN _ 22 SUB 22 are _ VB VBP VERB _ 16 SBAR 23 limited _ JJ JJ ADJ _ 22 PRD 24 . _ . . . _ 4 P 1 The _ DT DT DET _ 4 NMOD 2 oboist _ NN NN NOUN _ 4 NMOD 3 Heinz _ NN NNP NOUN _ 4 NMOD 4 Holliger _ NN NNP NOUN _ 5 SUB 5 has _ VB VBZ VERB _ 15 VMOD 6 taken _ VB VBN VERB _ 5 VC 7 a _ DT DT DET _ 9 NMOD 8 hard _ JJ JJ ADJ _ 9 NMOD 9 line _ NN NN NOUN _ 6 OBJ 10 about _ IN IN ADP _ 6 VMOD 11 the _ DT DT DET _ 12 NMOD 12 problem _ NN NN NOUN _ 10 PMOD 13 : _ : : . _ 15 P 14 He _ PR PRP PRON _ 15 SUB 15 commissions _ VB VBZ VERB _ 0 ROOT 16 and _ CC CC CONJ _ 15 VMOD 17 splendidly _ RB RB ADV _ 18 VMOD 18 interprets _ VB VBZ VERB _ 15 VMOD 19 fearsome _ JJ JJ ADJ _ 21 NMOD 20 contemporary _ JJ JJ ADJ _ 21 NMOD 21 scores _ NN NNS NOUN _ 15 VMOD 22 and _ CC CC CONJ _ 15 VMOD 23 does _ VB VBZ VERB _ 15 VMOD 24 some _ DT DT DET _ 25 NMOD 25 conducting _ NN NN NOUN _ 23 OBJ 26 , _ , , . _ 15 P 27 so _ IN IN ADP _ 15 VMOD 28 he _ PR PRP PRON _ 29 SUB 29 does _ VB VBZ VERB _ 27 SBAR 30 n't _ RB RB ADV _ 29 VMOD 31 have _ VB VB VERB _ 29 VC 32 to _ TO TO PRT _ 33 VMOD 33 play _ VB VB VERB _ 31 VMOD 34 the _ DT DT DET _ 39 NMOD 35 same _ JJ JJ ADJ _ 39 NMOD 36 Mozart _ NN NNP NOUN _ 39 NMOD 37 and _ CC CC CONJ _ 39 NMOD 38 Strauss _ NN NNP NOUN _ 39 NMOD 39 concertos _ NN NNS NOUN _ 33 OBJ 40 over _ RB RB ADV _ 33 VMOD 41 and _ CC CC CONJ _ 40 AMOD 42 over _ RB RB ADV _ 40 AMOD 43 again _ RB RB ADV _ 33 VMOD 44 . _ . . . _ 15 P 1 Richard _ NN NNP NOUN _ 2 NMOD 2 Stoltzman _ NN NNP NOUN _ 3 SUB 3 has _ VB VBZ VERB _ 0 ROOT 4 taken _ VB VBN VERB _ 3 VC 5 a _ DT DT DET _ 10 NMOD 6 gentler _ JJ JJR ADJ _ 10 NMOD 7 , _ , , . _ 10 P 8 more _ RB RBR ADV _ 9 AMOD 9 audience-friendly _ JJ JJ ADJ _ 10 NMOD 10 approach _ NN NN NOUN _ 4 OBJ 11 . _ . . . _ 3 P 1 Years _ NN NNS NOUN _ 2 AMOD 2 ago _ RB RB ADV _ 5 VMOD 3 , _ , , . _ 5 P 4 he _ PR PRP PRON _ 5 SUB 5 collaborated _ VB VBD VERB _ 0 ROOT 6 with _ IN IN ADP _ 5 VMOD 7 the _ DT DT DET _ 10 NMOD 8 new _ JJ JJ ADJ _ 10 NMOD 9 music _ NN NN NOUN _ 10 NMOD 10 gurus _ NN NNS NOUN _ 15 NMOD 11 Peter _ NN NNP NOUN _ 12 NMOD 12 Serkin _ NN NNP NOUN _ 15 NMOD 13 and _ CC CC CONJ _ 15 NMOD 14 Fred _ NN NNP NOUN _ 15 NMOD 15 Sherry _ NN NNP NOUN _ 6 PMOD 16 in _ IN IN ADP _ 5 VMOD 17 the _ DT DT DET _ 21 NMOD 18 very _ RB RB ADV _ 19 AMOD 19 countercultural _ JJ JJ ADJ _ 21 NMOD 20 chamber _ NN NN NOUN _ 21 NMOD 21 group _ NN NN NOUN _ 22 NMOD 22 Tashi _ NN NNP NOUN _ 16 PMOD 23 , _ , , . _ 22 P 24 which _ WD WDT DET _ 22 NMOD 25 won _ VB VBD VERB _ 24 SBAR 26 audiences _ NN NNS NOUN _ 25 OBJ 27 over _ RP RP PRT _ 25 VMOD 28 to _ TO TO PRT _ 27 AMOD 29 dreaded _ JJ JJ ADJ _ 31 NMOD 30 contemporary _ JJ JJ ADJ _ 31 NMOD 31 scores _ NN NNS NOUN _ 28 PMOD 32 like _ IN IN ADP _ 31 NMOD 33 Messiaen _ NN NNP NOUN _ 34 NMOD 34 's _ PO POS PRT _ 36 NMOD 35 `` _ `` `` . _ 36 P 36 Quartet _ NN NN NOUN _ 32 PMOD 37 for _ IN IN ADP _ 36 NMOD 38 the _ DT DT DET _ 39 NMOD 39 End _ NN NN NOUN _ 37 PMOD 40 of _ IN IN ADP _ 39 NMOD 41 Time _ NN NN NOUN _ 40 PMOD 42 . _ . . . _ 5 P 43 '' _ '' '' . _ 5 P 1 Today _ NN NN NOUN _ 6 VMOD 2 , _ , , . _ 6 P 3 the _ DT DT DET _ 5 NMOD 4 pixie-like _ JJ JJ ADJ _ 5 NMOD 5 clarinetist _ NN NN NOUN _ 6 SUB 6 has _ VB VBZ VERB _ 0 ROOT 7 mostly _ RB RB ADV _ 6 VMOD 8 dropped _ VB VBN VERB _ 6 VC 9 the _ DT DT DET _ 11 NMOD 10 missionary _ JJ JJ ADJ _ 11 NMOD 11 work _ NN NN NOUN _ 8 OBJ 12 ( _ -L -LRB- . _ 13 P 13 though _ IN IN ADP _ 8 VMOD 14 a _ DT DT DET _ 15 NMOD 15 touch _ NN NN NOUN _ 21 SUB 16 of _ IN IN ADP _ 15 NMOD 17 the _ DT DT DET _ 19 NMOD 18 old _ JJ JJ ADJ _ 19 NMOD 19 Tashi _ NN NNP NOUN _ 16 PMOD 20 still _ RB RB ADV _ 21 VMOD 21 survives _ VB VBZ VERB _ 13 SBAR 22 ) _ -R -RRB- . _ 13 P 23 and _ CC CC CONJ _ 6 VMOD 24 now _ RB RB ADV _ 6 VMOD 25 goes _ VB VBZ VERB _ 6 VMOD 26 on _ IN IN ADP _ 25 VMOD 27 the _ DT DT DET _ 28 NMOD 28 road _ NN NN NOUN _ 26 PMOD 29 with _ IN IN ADP _ 25 VMOD 30 piano _ NN NN NOUN _ 40 NMOD 31 , _ , , . _ 40 P 32 bass _ NN NN NOUN _ 40 NMOD 33 , _ , , . _ 40 P 34 a _ DT DT DET _ 36 NMOD 35 slide _ NN NN NOUN _ 36 NMOD 36 show _ NN NN NOUN _ 40 NMOD 37 , _ , , . _ 40 P 38 and _ CC CC CONJ _ 40 NMOD 39 a _ DT DT DET _ 40 NMOD 40 repertoire _ NN NN NOUN _ 29 PMOD 41 that _ WD WDT DET _ 40 NMOD 42 ranges _ VB VBZ VERB _ 41 SBAR 43 from _ IN IN ADP _ 42 VMOD 44 light _ JJ JJ ADJ _ 45 NMOD 45 classical _ JJ JJ ADJ _ 43 PMOD 46 to _ TO TO PRT _ 43 PMOD 47 light _ JJ JJ ADJ _ 48 NMOD 48 jazz _ NN NN NOUN _ 46 PMOD 49 to _ TO TO PRT _ 43 PMOD 50 light _ JJ JJ ADJ _ 51 NMOD 51 pop _ NN NN NOUN _ 49 PMOD 52 , _ , , . _ 42 P 53 with _ IN IN ADP _ 42 VMOD 54 a _ DT DT DET _ 57 NMOD 55 few _ JJ JJ ADJ _ 57 NMOD 56 notable _ JJ JJ ADJ _ 57 NMOD 57 exceptions _ NN NNS NOUN _ 53 PMOD 58 . _ . . . _ 6 P 1 Just _ RB RB ADV _ 3 NMOD 2 the _ DT DT DET _ 3 NMOD 3 thing _ NN NN NOUN _ 0 ROOT 4 for _ IN IN ADP _ 3 DEP 5 the _ DT DT DET _ 7 NMOD 6 Vivaldi-at-brunch _ JJ JJ ADJ _ 7 NMOD 7 set _ NN NN NOUN _ 11 NMOD 8 , _ , , . _ 11 P 9 the _ DT DT DET _ 11 NMOD 10 yuppie _ NN NN NOUN _ 11 NMOD 11 audience _ NN NN NOUN _ 4 PMOD 12 that _ WD WDT DET _ 11 NMOD 13 has _ VB VBZ VERB _ 12 SBAR 14 embraced _ VB VBN VERB _ 13 VC 15 New _ NN NNP NOUN _ 16 NMOD 16 Age _ NN NNP NOUN _ 14 OBJ 17 as _ IN IN ADP _ 14 VMOD 18 its _ PR PRP$ PRON _ 22 NMOD 19 very _ RB RB ADV _ 20 AMOD 20 own _ JJ JJ ADJ _ 22 NMOD 21 easy _ JJ JJ ADJ _ 22 NMOD 22 listening _ NN NN NOUN _ 17 PMOD 23 . _ . . . _ 3 P 1 But _ CC CC CONJ _ 3 VMOD 2 you _ PR PRP PRON _ 3 SUB 3 ca _ MD MD VERB _ 0 ROOT 4 n't _ RB RB ADV _ 3 VMOD 5 dismiss _ VB VB VERB _ 3 VC 6 Mr. _ NN NNP NOUN _ 8 NMOD 7 Stoltzman _ NN NNP NOUN _ 8 NMOD 8 's _ PO POS PRT _ 9 NMOD 9 music _ NN NN NOUN _ 12 NMOD 10 or _ CC CC CONJ _ 12 NMOD 11 his _ PR PRP$ PRON _ 12 NMOD 12 motives _ NN NNS NOUN _ 5 OBJ 13 as _ RB RB ADV _ 5 VMOD 14 merely _ RB RB ADV _ 17 AMOD 15 commercial _ JJ JJ ADJ _ 17 AMOD 16 and _ CC CC CONJ _ 17 AMOD 17 lightweight _ JJ JJ ADJ _ 13 PMOD 18 . _ . . . _ 3 P 1 He _ PR PRP PRON _ 2 SUB 2 believes _ VB VBZ VERB _ 10 VMOD 3 in _ IN IN ADP _ 2 VMOD 4 what _ WP WP PRON _ 3 PMOD 5 he _ PR PRP PRON _ 6 SUB 6 plays _ VB VBZ VERB _ 4 SBAR 7 , _ , , . _ 10 P 8 and _ CC CC CONJ _ 10 VMOD 9 he _ PR PRP PRON _ 10 SUB 10 plays _ VB VBZ VERB _ 0 ROOT 11 superbly _ RB RB ADV _ 10 VMOD 12 . _ . . . _ 10 P 1 His _ PR PRP$ PRON _ 3 NMOD 2 recent _ JJ JJ ADJ _ 3 NMOD 3 appearance _ NN NN NOUN _ 16 SUB 4 at _ IN IN ADP _ 3 NMOD 5 the _ DT DT DET _ 7 NMOD 6 Metropolitan _ NN NNP NOUN _ 7 NMOD 7 Museum _ NN NNP NOUN _ 4 PMOD 8 , _ , , . _ 3 P 9 dubbed _ VB VBN VERB _ 3 NMOD 10 `` _ `` `` . _ 13 P 11 A _ DT DT DET _ 13 NMOD 12 Musical _ JJ JJ ADJ _ 13 NMOD 13 Odyssey _ NN NN NOUN _ 9 VMOD 14 , _ , , . _ 13 P 15 '' _ '' '' . _ 13 P 16 was _ VB VBD VERB _ 0 ROOT 17 a _ DT DT DET _ 18 NMOD 18 case _ NN NN NOUN _ 16 PRD 19 in _ IN IN ADP _ 18 NMOD 20 point _ NN NN NOUN _ 19 PMOD 21 . _ . . . _ 16 P 1 It _ PR PRP PRON _ 2 SUB 2 felt _ VB VBD VERB _ 0 ROOT 3 more _ RB RBR ADV _ 4 PMOD 4 like _ IN IN ADP _ 2 VMOD 5 a _ DT DT DET _ 6 NMOD 6 party _ NN NN NOUN _ 13 NMOD 7 , _ , , . _ 13 P 8 or _ CC CC CONJ _ 13 NMOD 9 a _ DT DT DET _ 13 NMOD 10 highly _ RB RB ADV _ 11 AMOD 11 polished _ JJ JJ ADJ _ 13 NMOD 12 jam _ NN NN NOUN _ 13 NMOD 13 session _ NN NN NOUN _ 4 PMOD 14 with _ IN IN ADP _ 13 NMOD 15 a _ DT DT DET _ 17 NMOD 16 few _ JJ JJ ADJ _ 17 NMOD 17 friends _ NN NNS NOUN _ 14 PMOD 18 , _ , , . _ 13 P 19 than _ IN IN ADP _ 4 PMOD 20 a _ DT DT DET _ 22 NMOD 21 classical _ JJ JJ ADJ _ 22 NMOD 22 concert _ NN NN NOUN _ 19 PMOD 23 . _ . . . _ 2 P 1 Clad _ VB VBN VERB _ 12 VMOD 2 in _ IN IN ADP _ 1 VMOD 3 his _ PR PRP$ PRON _ 7 NMOD 4 trademark _ NN NN NOUN _ 7 NMOD 5 black _ JJ JJ ADJ _ 7 NMOD 6 velvet _ NN NN NOUN _ 7 NMOD 7 suit _ NN NN NOUN _ 2 PMOD 8 , _ , , . _ 12 P 9 the _ DT DT DET _ 11 NMOD 10 soft-spoken _ JJ JJ ADJ _ 11 NMOD 11 clarinetist _ NN NN NOUN _ 12 SUB 12 announced _ VB VBD VERB _ 0 ROOT 13 that _ IN IN ADP _ 38 DEP 14 his _ PR PRP$ PRON _ 16 NMOD 15 new _ JJ JJ ADJ _ 16 NMOD 16 album _ NN NN NOUN _ 20 NMOD 17 , _ , , . _ 20 P 18 `` _ `` `` . _ 20 P 19 Inner _ JJ JJ ADJ _ 20 NMOD 20 Voices _ NN NNS NOUN _ 23 SUB 21 , _ , , . _ 20 P 22 '' _ '' '' . _ 20 P 23 had _ VB VBD VERB _ 13 SBAR 24 just _ RB RB ADV _ 23 VMOD 25 been _ VB VBN VERB _ 23 VC 26 released _ VB VBN VERB _ 25 VC 27 , _ , , . _ 38 P 28 that _ IN IN ADP _ 38 DEP 29 his _ PR PRP$ PRON _ 30 NMOD 30 family _ NN NN NOUN _ 31 SUB 31 was _ VB VBD VERB _ 28 SBAR 32 in _ IN IN ADP _ 31 VMOD 33 the _ DT DT DET _ 35 NMOD 34 front _ NN NN NOUN _ 35 NMOD 35 row _ NN NN NOUN _ 32 PMOD 36 , _ , , . _ 38 P 37 and _ CC CC CONJ _ 38 DEP 38 that _ IN IN ADP _ 12 VMOD 39 it _ PR PRP PRON _ 40 SUB 40 was _ VB VBD VERB _ 48 VMOD 41 his _ PR PRP$ PRON _ 43 NMOD 42 mother _ NN NN NOUN _ 43 NMOD 43 's _ PO POS PRT _ 44 NMOD 44 birthday _ NN NN NOUN _ 40 PRD 45 , _ , , . _ 48 P 46 so _ RB RB ADV _ 48 VMOD 47 he _ PR PRP PRON _ 48 SUB 48 was _ VB VBD VERB _ 38 SBAR 49 going _ VB VBG VERB _ 48 VC 50 to _ TO TO PRT _ 51 VMOD 51 play _ VB VB VERB _ 49 VMOD 52 her _ PR PRP$ PRON _ 54 NMOD 53 favorite _ JJ JJ ADJ _ 54 NMOD 54 tune _ NN NN NOUN _ 51 OBJ 55 from _ IN IN ADP _ 54 NMOD 56 the _ DT DT DET _ 57 NMOD 57 record _ NN NN NOUN _ 55 PMOD 58 . _ . . . _ 12 P 1 He _ PR PRP PRON _ 2 SUB 2 launched _ VB VBD VERB _ 0 ROOT 3 into _ IN IN ADP _ 2 VMOD 4 Saint-Saens _ NN NNP NOUN _ 5 NMOD 5 's _ PO POS PRT _ 8 NMOD 6 `` _ `` `` . _ 8 P 7 The _ DT DT DET _ 8 NMOD 8 Swan _ NN NN NOUN _ 21 NMOD 9 '' _ '' '' . _ 8 P 10 from _ IN IN ADP _ 8 NMOD 11 `` _ `` `` . _ 10 P 12 Carnival _ NN NN NOUN _ 10 PMOD 13 of _ IN IN ADP _ 12 NMOD 14 the _ DT DT DET _ 15 NMOD 15 Animals _ NN NNS NOUN _ 13 PMOD 16 , _ , , . _ 21 P 17 '' _ '' '' . _ 21 P 18 a _ DT DT DET _ 21 NMOD 19 favorite _ JJ JJ ADJ _ 21 NMOD 20 encore _ NN NN NOUN _ 21 NMOD 21 piece _ NN NN NOUN _ 3 PMOD 22 for _ IN IN ADP _ 21 NMOD 23 cellists _ NN NNS NOUN _ 22 PMOD 24 , _ , , . _ 21 P 25 with _ IN IN ADP _ 2 VMOD 26 lovely _ JJ JJ ADJ _ 29 NMOD 27 , _ , , . _ 29 P 28 glossy _ JJ JJ ADJ _ 29 NMOD 29 tone _ NN NN NOUN _ 32 NMOD 30 and _ CC CC CONJ _ 32 NMOD 31 no _ DT DT DET _ 32 NMOD 32 bite _ NN NN NOUN _ 25 PMOD 33 . _ . . . _ 2 P 1 Then _ RB RB ADV _ 16 VMOD 2 , _ , , . _ 16 P 3 as _ IN IN ADP _ 4 DEP 4 if _ IN IN ADP _ 16 VMOD 5 to _ TO TO PRT _ 6 VMOD 6 show _ VB VB VERB _ 4 SBAR 7 that _ IN IN ADP _ 6 VMOD 8 he _ PR PRP PRON _ 9 SUB 9 could _ MD MD VERB _ 7 SBAR 10 play _ VB VB VERB _ 9 VC 11 fast _ RB RB ADV _ 10 VMOD 12 as _ IN IN ADP _ 13 AMOD 13 well _ RB RB ADV _ 10 VMOD 14 , _ , , . _ 16 P 15 he _ PR PRP PRON _ 16 SUB 16 offered _ VB VBD VERB _ 0 ROOT 17 the _ DT DT DET _ 19 NMOD 18 second _ JJ JJ ADJ _ 19 NMOD 19 movement _ NN NN NOUN _ 31 NMOD 20 from _ IN IN ADP _ 19 NMOD 21 Saint-Saens _ NN NNP NOUN _ 22 NMOD 22 's _ PO POS PRT _ 23 NMOD 23 Sonata _ NN NN NOUN _ 20 PMOD 24 for _ IN IN ADP _ 23 NMOD 25 Clarinet _ NN NN NOUN _ 24 PMOD 26 , _ , , . _ 31 P 27 a _ DT DT DET _ 31 NMOD 28 whimsical _ JJ JJ ADJ _ 31 NMOD 29 , _ , , . _ 31 P 30 puckish _ JJ JJ ADJ _ 31 NMOD 31 tidbit _ NN NN NOUN _ 16 OBJ 32 that _ WD WDT DET _ 31 NMOD 33 reflected _ VB VBD VERB _ 32 SBAR 34 the _ DT DT DET _ 36 NMOD 35 flip _ JJ JJ ADJ _ 36 NMOD 36 side _ NN NN NOUN _ 33 OBJ 37 of _ IN IN ADP _ 36 NMOD 38 the _ DT DT DET _ 40 NMOD 39 Stoltzman _ NN NNP NOUN _ 40 NMOD 40 personality _ NN NN NOUN _ 37 PMOD 41 . _ . . . _ 16 P 1 And _ CC CC CONJ _ 4 VMOD 2 so _ RB RB ADV _ 4 VMOD 3 it _ PR PRP PRON _ 4 SUB 4 went _ VB VBD VERB _ 0 ROOT 5 through _ IN IN ADP _ 4 VMOD 6 the _ DT DT DET _ 8 NMOD 7 first _ JJ JJ ADJ _ 8 NMOD 8 half _ NN NN NOUN _ 5 PMOD 9 : _ : : . _ 4 P 10 an _ DT DT DET _ 13 NMOD 11 ingeniously _ RB RB ADV _ 12 AMOD 12 chosen _ VB VBN VERB _ 13 NMOD 13 potpourri _ NN NN NOUN _ 4 DEP 14 of _ IN IN ADP _ 13 NMOD 15 pieces _ NN NNS NOUN _ 23 NMOD 16 , _ , , . _ 23 P 17 none _ NN NN NOUN _ 23 NMOD 18 longer _ JJ JJR ADJ _ 17 NMOD 19 than _ IN IN ADP _ 18 AMOD 20 five _ CD CD NUM _ 21 NMOD 21 minutes _ NN NNS NOUN _ 19 PMOD 22 , _ , , . _ 23 P 23 none _ NN NN NOUN _ 14 PMOD 24 that _ WD WDT DET _ 23 NMOD 25 would _ MD MD VERB _ 24 SBAR 26 disturb _ VB VB VERB _ 25 VC 27 or _ CC CC CONJ _ 26 VMOD 28 challenge _ VB VB VERB _ 26 VMOD 29 a _ DT DT DET _ 30 NMOD 30 listener _ NN NN NOUN _ 26 OBJ 31 . _ . . . _ 4 P 1 Mr. _ NN NNP NOUN _ 2 NMOD 2 Stoltzman _ NN NNP NOUN _ 3 SUB 3 introduced _ VB VBD VERB _ 0 ROOT 4 his _ PR PRP$ PRON _ 5 NMOD 5 colleagues _ NN NNS NOUN _ 22 NMOD 6 : _ : : . _ 22 P 7 Bill _ NN NNP NOUN _ 8 NMOD 8 Douglas _ NN NNP NOUN _ 14 NMOD 9 , _ , , . _ 14 P 10 pianist\/bassoonist\/composer _ NN NN NOUN _ 14 NMOD 11 and _ CC CC CONJ _ 14 NMOD 12 an _ DT DT DET _ 14 NMOD 13 old _ JJ JJ ADJ _ 14 NMOD 14 buddy _ NN NN NOUN _ 22 NMOD 15 from _ IN IN ADP _ 14 NMOD 16 Yale _ NN NNP NOUN _ 15 PMOD 17 , _ , , . _ 22 P 18 and _ CC CC CONJ _ 22 NMOD 19 jazz _ NN NN NOUN _ 22 NMOD 20 bassist _ NN NN NOUN _ 22 NMOD 21 Eddie _ NN NNP NOUN _ 22 NMOD 22 Gomez _ NN NNP NOUN _ 3 OBJ 23 . _ . . . _ 3 P 1 An _ DT DT DET _ 3 NMOD 2 improvisational _ JJ JJ ADJ _ 3 NMOD 3 section _ NN NN NOUN _ 4 SUB 4 was _ VB VBD VERB _ 0 ROOT 5 built _ VB VBN VERB _ 4 VC 6 around _ IN IN ADP _ 5 VMOD 7 pieces _ NN NNS NOUN _ 6 PMOD 8 by _ IN IN ADP _ 7 NMOD 9 Mr. _ NN NNP NOUN _ 10 NMOD 10 Douglas _ NN NNP NOUN _ 8 PMOD 11 , _ , , . _ 5 P 12 beginning _ NN NN NOUN _ 5 VMOD 13 with _ IN IN ADP _ 12 PMOD 14 `` _ `` `` . _ 23 P 15 Golden _ JJ JJ ADJ _ 16 NMOD 16 Rain _ NN NN NOUN _ 23 NMOD 17 , _ , , . _ 23 P 18 '' _ '' '' . _ 23 P 19 a _ DT DT DET _ 23 NMOD 20 lilting _ JJ JJ ADJ _ 23 NMOD 21 , _ , , . _ 23 P 22 laid-back _ JJ JJ ADJ _ 23 NMOD 23 lead _ NN NN NOUN _ 13 PMOD 24 in _ IN IN ADP _ 23 NMOD 25 to _ TO TO PRT _ 24 PMOD 26 the _ DT DT DET _ 29 NMOD 27 uptempo _ JJ JJ ADJ _ 29 NMOD 28 `` _ `` `` . _ 29 P 29 Sky _ NN NN NOUN _ 25 PMOD 30 , _ , , . _ 29 P 31 '' _ '' '' . _ 29 P 32 which _ WD WDT DET _ 29 NMOD 33 gave _ VB VBD VERB _ 32 SBAR 34 Mr. _ NN NNP NOUN _ 35 NMOD 35 Stoltzman _ NN NNP NOUN _ 33 OBJ 36 the _ DT DT DET _ 37 NMOD 37 opportunity _ NN NN NOUN _ 33 OBJ 38 to _ TO TO PRT _ 39 VMOD 39 wail _ VB VB VERB _ 37 NMOD 40 in _ IN IN ADP _ 39 VMOD 41 a _ DT DT DET _ 43 NMOD 42 high _ JJ JJ ADJ _ 43 NMOD 43 register _ NN NN NOUN _ 40 PMOD 44 and _ CC CC CONJ _ 39 VMOD 45 show _ VB VB VERB _ 39 VMOD 46 off _ RP RP PRT _ 45 VMOD 47 his _ PR PRP$ PRON _ 49 NMOD 48 fleet _ JJ JJ ADJ _ 49 NMOD 49 fingers _ NN NNS NOUN _ 45 OBJ 50 . _ . . . _ 4 P 1 Bach _ NN NNP NOUN _ 2 NMOD 2 's _ PO POS PRT _ 4 NMOD 3 `` _ `` `` . _ 4 P 4 Air _ NN NN NOUN _ 6 SUB 5 '' _ '' '' . _ 4 P 6 followed _ VB VBD VERB _ 0 ROOT 7 . _ . . . _ 6 P 1 Mr. _ NN NNP NOUN _ 2 NMOD 2 Stoltzman _ NN NNP NOUN _ 3 SUB 3 tied _ VB VBD VERB _ 0 ROOT 4 the _ DT DT DET _ 5 NMOD 5 composer _ NN NN NOUN _ 3 OBJ 6 in _ RP RP PRT _ 3 VMOD 7 by _ IN IN ADP _ 3 VMOD 8 proclaiming _ VB VBG VERB _ 7 PMOD 9 him _ PR PRP PRON _ 13 SUB 10 `` _ `` `` . _ 13 P 11 the _ DT DT DET _ 13 NMOD 12 great _ JJ JJ ADJ _ 13 NMOD 13 improviser _ NN NN NOUN _ 8 VMOD 14 of _ IN IN ADP _ 13 NMOD 15 the _ DT DT DET _ 17 NMOD 16 18th _ JJ JJ ADJ _ 17 NMOD 17 century _ NN NN NOUN _ 14 PMOD 18 , _ , , . _ 3 P 19 '' _ '' '' . _ 3 P 20 and _ CC CC CONJ _ 3 VMOD 21 then _ RB RB ADV _ 3 VMOD 22 built _ VB VBD VERB _ 3 VMOD 23 on _ IN IN ADP _ 22 VMOD 24 the _ DT DT DET _ 25 NMOD 25 image _ NN NN NOUN _ 23 PMOD 26 by _ IN IN ADP _ 22 VMOD 27 joining _ VB VBG VERB _ 26 PMOD 28 with _ IN IN ADP _ 27 VMOD 29 Mr. _ NN NNP NOUN _ 30 NMOD 30 Douglas _ NN NNP NOUN _ 28 PMOD 31 in _ IN IN ADP _ 27 VMOD 32 some _ DT DT DET _ 35 NMOD 33 Bach _ NN NNP NOUN _ 35 NMOD 34 two-part _ JJ JJ ADJ _ 35 NMOD 35 inventions _ NN NNS NOUN _ 31 PMOD 36 , _ , , . _ 35 P 37 cleverly _ RB RB ADV _ 38 VMOD 38 arranged _ VB VBN VERB _ 35 NMOD 39 for _ IN IN ADP _ 38 VMOD 40 clarinet _ NN NN NOUN _ 42 NMOD 41 and _ CC CC CONJ _ 42 NMOD 42 bassoon _ NN NN NOUN _ 39 PMOD 43 by _ IN IN ADP _ 38 VMOD 44 Mr. _ NN NNP NOUN _ 45 NMOD 45 Douglas _ NN NNP NOUN _ 43 PMOD 46 . _ . . . _ 3 P 1 Keeping _ VB VBG VERB _ 9 VMOD 2 the _ DT DT DET _ 3 NMOD 3 mood _ NN NN NOUN _ 4 SUB 4 light _ JJ JJ ADJ _ 1 VMOD 5 , _ , , . _ 9 P 6 the _ DT DT DET _ 7 NMOD 7 two _ CD CD NUM _ 9 SUB 8 then _ RB RB ADV _ 9 VMOD 9 chanted _ VB VBD VERB _ 0 ROOT 10 and _ CC CC CONJ _ 9 VMOD 11 chortled _ VB VBD VERB _ 9 VMOD 12 their _ PR PRP$ PRON _ 13 NMOD 13 way _ NN NN NOUN _ 9 OBJ 14 through _ IN IN ADP _ 9 VMOD 15 some _ DT DT DET _ 17 NMOD 16 murderous _ JJ JJ ADJ _ 17 NMOD 17 polyrhythms _ NN NNS NOUN _ 14 PMOD 18 , _ , , . _ 17 P 19 devised _ VB VBN VERB _ 17 NMOD 20 by _ IN IN ADP _ 19 VMOD 21 Mr. _ NN NNP NOUN _ 22 NMOD 22 Douglas _ NN NNP NOUN _ 20 PMOD 23 as _ IN IN ADP _ 19 VMOD 24 an _ DT DT DET _ 25 NMOD 25 alternative _ NN NN NOUN _ 23 PMOD 26 to _ TO TO PRT _ 25 NMOD 27 Hindemith _ NN NNP NOUN _ 28 NMOD 28 's _ PO POS PRT _ 31 NMOD 29 dry _ JJ JJ ADJ _ 31 NMOD 30 theory-teaching _ VB VBG VERB _ 31 NMOD 31 techniques _ NN NNS NOUN _ 26 PMOD 32 , _ , , . _ 9 P 33 and _ CC CC CONJ _ 9 VMOD 34 then _ RB RB ADV _ 9 VMOD 35 , _ , , . _ 9 P 36 with _ IN IN ADP _ 40 VMOD 37 Mr. _ NN NNP NOUN _ 38 NMOD 38 Gomez _ NN NNP NOUN _ 36 PMOD 39 , _ , , . _ 40 P 40 soared _ VB VBD VERB _ 9 VMOD 41 and _ CC CC CONJ _ 40 VMOD 42 improvised _ VB VBD VERB _ 40 VMOD 43 on _ IN IN ADP _ 42 VMOD 44 the _ DT DT DET _ 46 NMOD 45 composer _ NN NN NOUN _ 46 NMOD 46 's _ PO POS PRT _ 50 NMOD 47 tight _ JJ JJ ADJ _ 50 NMOD 48 `` _ `` `` . _ 50 P 49 Bebop _ NN NNP NOUN _ 50 NMOD 50 Etudes _ NN NNP NOUN _ 43 PMOD 51 . _ . . . _ 9 P 52 '' _ '' '' . _ 9 P 1 The _ DT DT DET _ 2 NMOD 2 end _ NN NN NOUN _ 10 SUB 3 of _ IN IN ADP _ 2 NMOD 4 the _ DT DT DET _ 6 NMOD 5 first _ JJ JJ ADJ _ 6 NMOD 6 half _ NN NN NOUN _ 3 PMOD 7 , _ , , . _ 10 P 8 however _ RB RB ADV _ 10 VMOD 9 , _ , , . _ 10 P 10 brought _ VB VBD VERB _ 0 ROOT 11 what _ WP WP PRON _ 25 NMOD 12 the _ DT DT DET _ 14 NMOD 13 standing-room-only _ JJ JJ ADJ _ 14 NMOD 14 crowd _ NN NN NOUN _ 15 SUB 15 seemed _ VB VBD VERB _ 11 SBAR 16 to _ TO TO PRT _ 17 VMOD 17 be _ VB VB VERB _ 15 VMOD 18 waiting _ VB VBG VERB _ 17 VC 19 for _ IN IN ADP _ 18 VMOD 20 : _ : : . _ 25 P 21 the _ DT DT DET _ 23 NMOD 22 pop _ NN NN NOUN _ 23 NMOD 23 singer _ NN NN NOUN _ 25 NMOD 24 Judy _ NN NNP NOUN _ 25 NMOD 25 Collins _ NN NNP NOUN _ 10 OBJ 26 , _ , , . _ 25 P 27 who _ WP WP PRON _ 25 NMOD 28 appears _ VB VBZ VERB _ 27 SBAR 29 on _ IN IN ADP _ 28 VMOD 30 `` _ `` `` . _ 29 P 31 Inner _ JJ JJ ADJ _ 32 NMOD 32 Voices _ NN NNS NOUN _ 29 PMOD 33 . _ . . . _ 10 P 34 '' _ '' '' . _ 10 P vowpal-wabbit-8.5.0.dfsg1/demo/dna/000077500000000000000000000000001323713302100167665ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/demo/dna/.gitignore000066400000000000000000000000721323713302100207550ustar00rootroot00000000000000*.gz *.bz2 quaddna2vw *.model *cache* *.predictions *.pid vowpal-wabbit-8.5.0.dfsg1/demo/dna/Makefile000066400000000000000000000012541323713302100204300ustar00rootroot00000000000000CXXFLAGS=-O3 .SECONDARY: all: @cat README clean: rm -f $(wildcard dna*.test.predictions dna*.model) %.check: @test -x "$$(which $*)" || { \ echo "ERROR: you need to install $*" 1>&2; \ exit 1; \ } dna_train.%.bz2: wget.check wget ftp://largescale.ml.tu-berlin.de/largescale/dna/dna_train.$*.bz2 quaddna2vw: quaddna2vw.cpp %.model: dna_train.lab.bz2 dna_train.dat.bz2 quaddna2vw paste.check bzcat.check zsh.check ./do-$*-multicore-train %.test.predictions: %.model dna_train.lab.bz2 dna_train.dat.bz2 quaddna2vw paste.check bzcat.check zsh.check ./do-test $< $@ %.perf: %.test.predictions perf.check perl.check zsh.check ./do-perf $< .PHONY: all clean vowpal-wabbit-8.5.0.dfsg1/demo/dna/README000066400000000000000000000035641323713302100176560ustar00rootroot00000000000000This is the Splice Site recognition dataset from the 2008 Pascal Large Scale Learning Challenge (http://largescale.ml.tu-berlin.de/summary/). === WARNINGS === * you need a beefy machine to comfortably run these demos 4 cores or more SSD or lots of RAM because the demo is I/O bound * the demos are intolerably slow under cygwin (pipes do not work well) === INSTRUCTIONS === * make dna.perf downloads the dna dataset, trains a 4-gram logistic regression in parallel on 4 cores and computes test set statistics disk space requirements: about 3 gigabytes training time requirements: about 7 cores for about 15 minutes ultimately I/O bound: bzcat is the limiting factor memory requirements: less than 512 megabytes results in APR of 0.512 * make dnann.perf same as dna.perf, but with additionally 1 neural network hidden node slower (by circa 60 seconds) but better results in APR of 0.532 * make dnasmash.perf as above but builds a better model uses 10 iteratons of parallel sgd with 4 neural network nodes disk space requirements: about 10 gigabytes additional 7 gigabytes of vw cache on top of original data training time requirements: 16 minutes to build cache over first (ever) pass subsequently, 6 minute per pass if you have SSD or enough RAM cache 10 passes = 60 minutes (x 6 cores) results in APR of 0.545 * make dnahogwild.perf same as dna.perf, but trained via lock-free multicore sgd ("hogwild") rather than parallel sgd + averaging nondeterministic, but a typical result is APR of 0.516 * make dnahogwildnn.perf same as dnann.perf, but trained via lock-free multicore sgd ("hogwild") rather than parallel sgd + averaging nondeterministic, but a typical result is APR of 0.536 vowpal-wabbit-8.5.0.dfsg1/demo/dna/do-dna-multicore-train000077500000000000000000000015771323713302100232040ustar00rootroot00000000000000#! /bin/zsh learner() { ./quaddna2vw | \ ../../vowpalwabbit/vw -f dna.model.$1 \ --loss_function logistic \ -b 18 -l 0.25 --adaptive --invariant \ --total $2 --node $1 --unique_id 0 --span_server localhost \ } set -e ../../cluster/spanning_tree dna_spanning_tree.pid paste -d' ' \ <(bzcat dna_train.lab.bz2) \ <(bzcat dna_train.dat.bz2) | \ tail -n +1000000 | \ ./map \ >(learner 0 4) \ >(learner 1 4 >/dev/null 2>/dev/null) \ >(learner 2 4 >/dev/null 2>/dev/null) \ >(learner 3 4 >/dev/null 2>/dev/null) kill $(cat dna_spanning_tree.pid) wait rm dna_spanning_tree.pid mv dna.model.0 dna.model rm -f dna.model.* vowpal-wabbit-8.5.0.dfsg1/demo/dna/do-dnahogwild-multicore-train000077500000000000000000000020701323713302100245470ustar00rootroot00000000000000#! /bin/zsh rm -f dnahogwild.model set -e nukeem() { \ trap - INT QUIT TERM pkill -9 -f 'vw.*--port 26543' } learner() { ./quaddna2vw | \ netcat localhost 26543 > /dev/null } { ../../vowpalwabbit/vw -f dnahogwild.model \ --loss_function logistic \ -b 18 -l 0.0625 --adaptive --invariant \ --daemon --num_children 4 --port 26543 2>&1 | \ perl -lane 'print $_ unless $c{$F[2]}++;' } & trap 'nukeem; exit 1' INT QUIT TERM while ! netcat -z localhost 26543 do sleep 1 done paste -d' ' \ <(bzcat dna_train.lab.bz2) \ <(bzcat dna_train.dat.bz2) | \ tail -n +1000000 | \ ./map \ >(learner) \ >(learner) \ >(learner) \ >(learner) pkill -f 'vw.*--port 26543' while test ! -s dnahogwild.model do sleep 1 done vowpal-wabbit-8.5.0.dfsg1/demo/dna/do-dnahogwildnn-multicore-train000077500000000000000000000020741323713302100251070ustar00rootroot00000000000000#! /bin/zsh rm -f dnahogwildnn.model set -e nukeem() { \ trap - INT QUIT TERM pkill -9 -f 'vw.*--port 26544' } learner() { ./quaddna2vw | \ netcat localhost 26544 > /dev/null } { ../../vowpalwabbit/vw -f dnahogwildnn.model \ --loss_function logistic --nn 1 --inpass \ -b 18 -l 0.015 --adaptive --invariant \ --daemon --num_children 4 --port 26544 2>&1 | \ perl -lane 'print $_ unless $c{$F[2]}++;' } & trap 'nukeem; exit 1' INT QUIT TERM while ! netcat -z localhost 26544 do sleep 1 done paste -d' ' \ <(bzcat dna_train.lab.bz2) \ <(bzcat dna_train.dat.bz2) | \ tail -n +1000000 | \ ./map \ >(learner) \ >(learner) \ >(learner) \ >(learner) pkill -f 'vw.*--port 26544' while test ! -s dnahogwildnn.model do sleep 1 done vowpal-wabbit-8.5.0.dfsg1/demo/dna/do-dnann-multicore-train000077500000000000000000000016321323713302100235300ustar00rootroot00000000000000#! /bin/zsh learner() { ./quaddna2vw | \ ../../vowpalwabbit/vw -f dnann.model.$1 \ --loss_function logistic \ -b 18 -l 0.06 --adaptive --invariant \ --total $2 --node $1 --unique_id 0 --span_server localhost \ --nn 1 --inpass } set -e ../../cluster/spanning_tree dna_spanning_tree.pid paste -d' ' \ <(bzcat dna_train.lab.bz2) \ <(bzcat dna_train.dat.bz2) | \ tail -n +1000000 | \ ./map \ >(learner 0 4) \ >(learner 1 4 >/dev/null 2>/dev/null) \ >(learner 2 4 >/dev/null 2>/dev/null) \ >(learner 3 4 >/dev/null 2>/dev/null) kill $(cat dna_spanning_tree.pid) wait rm dna_spanning_tree.pid mv dnann.model.0 dnann.model rm -f dnann.model.* vowpal-wabbit-8.5.0.dfsg1/demo/dna/do-dnasmash-multicore-train000077500000000000000000000017641323713302100242360ustar00rootroot00000000000000#! /bin/zsh learner() { ./quaddna2vw | \ ../../vowpalwabbit/vw -f dnasmash.model.$1 \ --loss_function logistic \ --compressed --cache_file dnacache.$1 \ -b 20 -l 0.06 --adaptive --invariant --passes 10 \ --total $2 --node $1 --unique_id 0 --span_server localhost \ --nn 4 --inpass } ../../cluster/spanning_tree dnasmash_spanning_tree.pid paste -d' ' \ <(bzcat dna_train.lab.bz2) \ <(bzcat dna_train.dat.bz2) | \ tail -n +1000000 | \ ./map \ >(learner 0 4) \ >(learner 1 4 >/dev/null 2>/dev/null) \ >(learner 2 4 >/dev/null 2>/dev/null) \ >(learner 3 4 >/dev/null 2>/dev/null) kill $(cat dnasmash_spanning_tree.pid) wait rm dnasmash_spanning_tree.pid mv dnasmash.model.0 dnasmash.model rm -f dnasmash.model.* vowpal-wabbit-8.5.0.dfsg1/demo/dna/do-perf000077500000000000000000000003051323713302100202460ustar00rootroot00000000000000#! /bin/zsh perf -APR -ACC -ROC -t 0 \ -files <(bzcat dna_train.lab.bz2 | head -n 1000000) $1 || { echo "ERROR: you probably need to recompile perf to handle larger data sets" 1>&2 exit 1 } vowpal-wabbit-8.5.0.dfsg1/demo/dna/do-test000077500000000000000000000003721323713302100202750ustar00rootroot00000000000000#! /bin/zsh paste -d' ' \ <(bzcat dna_train.lab.bz2) \ <(bzcat dna_train.dat.bz2) | \ head -n +1000000 | \ ./quaddna2vw | \ ../../vowpalwabbit/vw -t --loss_function logistic -i $1 -p $2 vowpal-wabbit-8.5.0.dfsg1/demo/dna/map000077500000000000000000000004371323713302100174750ustar00rootroot00000000000000#! /usr/bin/env perl use warnings; use strict; use IO::File; my @mappers = map { $_->autoflush (0); $_ } map { new IO::File $_, "w" or die "$_: $!" } @ARGV; while (defined ($_ = )) { my $fh = $mappers[$. % @mappers]; print $fh $_; } vowpal-wabbit-8.5.0.dfsg1/demo/dna/quaddna2vw.cpp000066400000000000000000000020311323713302100215420ustar00rootroot00000000000000#include #include namespace { using namespace std; unsigned int codec (const string::const_iterator& c) { return *c == 'A' ? 0 : *c == 'C' ? 1 : *c == 'G' ? 2 : 3; } } int main (void) { using namespace std; while (! cin.eof ()) { string line; string label; getline (cin, line); if (line.length ()) { string::iterator s = line.begin (); while (*s != ' ') { cout << *s; ++s; } string::const_iterator ppp = s + 1; string::const_iterator pp = ppp + 1; string::const_iterator p = pp + 1; unsigned int offset = 1; cout << " |f"; for (string::const_iterator c = p + 1; c != line.end (); ++ppp, ++pp, ++p, ++c) { unsigned int val = 64 * codec (ppp) + 16 * codec (pp) + 4 * codec (p) + codec (c); cout << " " << offset + val << ":1"; offset += 256; } cout << endl; } } return 0; } vowpal-wabbit-8.5.0.dfsg1/demo/entityrelation/000077500000000000000000000000001323713302100212765ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/demo/entityrelation/README.md000077500000000000000000000006711323713302100225640ustar00rootroot00000000000000Searn for Entity Relation Recognition ------------------------------------- This demo shows the performance of Searn on an entity-relation recognition task. For more details on the data set see http://cogcomp.cs.illinois.edu/page/resource_view/43 ### Instructions ### - `make er.perf`: downloads the preprocessed entity-relation dataset, trains a joint model for predicting entity and relation types and computes test set statistics. vowpal-wabbit-8.5.0.dfsg1/demo/mnist/000077500000000000000000000000001323713302100173565ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/demo/mnist/.gitignore000066400000000000000000000001231323713302100213420ustar00rootroot00000000000000*.gz *.model *.predictions *.swp extractfeatures extractpixels pixelngrams *cache* vowpal-wabbit-8.5.0.dfsg1/demo/mnist/Makefile000066400000000000000000000052311323713302100210170ustar00rootroot00000000000000.SECONDARY: CXXFLAGS=-O3 -Wall help: @cat README RAW = mnist mnisthogwild mnist8m PNG = mnistpng mnist11png mnist8mpng mnist8m11png raw : $(addsuffix .test.confusion, $(RAW)) png : $(addsuffix .test.confusion, $(PNG)) MNIST = mnist mnisthogwild mnistpng mnist11png 8M = mnist8m mnist8mpng mnist8m11png mnist : $(addsuffix .test.confusion, $(MNIST)) mnist8m : $(addsuffix .test.confusion, $(8M)) .PHONY : clean testclean %.test.confusion mnist mnist8m raw png %.check: @test -x "$$(which $*)" || { \ echo "ERROR: you need to install $*" 1>&2; \ exit 1; \ } testclean: rm -f $(wildcard *.test.predictions) clean: rm -f $(wildcard mnist8mpng.*) $(wildcard mnist8m11png.*) $(wildcard mnist11png.*) $(wildcard mnistpng.*) $(wildcard mnist8m.*) $(wildcard mnist.*) # http://leon.bottou.org/projects/infimnist infimnist.tar.gz : wget.check wget http://leon.bottou.org/_media/projects/$@ infimnist : infimnist.tar.gz tar xvfz $< make -C $@ train8m-labels-idx1-ubyte.gz: infimnist (cd infimnist && ./infimnist lab 10000 8109999) | gzip > $@ train8m-images-idx3-ubyte.gz: infimnist (cd infimnist && ./infimnist pat 10000 8109999) | gzip > $@ train-%.gz: wget.check wget http://yann.lecun.com/exdb/mnist/train-$*.gz t10k-%.gz: wget.check wget http://yann.lecun.com/exdb/mnist/t10k-$*.gz pixelngrams: pixelngrams.cpp extractpixels: extractpixels.cpp extractfeatures: extractfeatures.cpp %png.model: train-labels-idx1-ubyte.gz train-images-idx3-ubyte.gz extractpixels pixelngrams zsh.check perl.check paste.check ./do-$*png-train mnist.model: train-labels-idx1-ubyte.gz train-images-idx3-ubyte.gz extractfeatures zsh.check perl.check paste.check ./do-mnist-train mnisthogwild.model: train-labels-idx1-ubyte.gz train-images-idx3-ubyte.gz extractfeatures zsh.check perl.check paste.check ./do-mnist-hogwild-train mnist8m.model: train8m-labels-idx1-ubyte.gz train8m-images-idx3-ubyte.gz extractfeatures zsh.check perl.check paste.check ./do-mnist8m-train %png.test.predictions: %png.model t10k-labels-idx1-ubyte.gz t10k-images-idx3-ubyte.gz zsh.check perl.check paste.check ./do-pixelngram-test $*png %.test.predictions: %.model t10k-labels-idx1-ubyte.gz t10k-images-idx3-ubyte.gz zsh.check perl.check paste.check ./do-test $* CONFUSION='++$$n; \ ++$$c if $$F[0] != $$F[1]; \ ++$$m{"@{[int($$F[0])]}:$$F[1]"}; } { \ print "$* test errors: $$c out of $$n"; \ print "confusion matrix (rows = truth, columns = prediction):"; \ foreach $$true (1 .. 10) { \ print join "\t", map { $$m{"$$true:$$_"} || 0 } (1 .. 10); \ }' %.test.confusion: %.test.predictions perl.check @perl -lane $(CONFUSION) $*.test.predictions vowpal-wabbit-8.5.0.dfsg1/demo/mnist/README000066400000000000000000000061061323713302100202410ustar00rootroot00000000000000mnist (http://yann.lecun.com/exdb/mnist/) is the traditional set for testing neural network implementations. mnist8m (http://leon.bottou.org/papers/loosli-canu-bottou-2006) is a variant of the original mnist training set augmented with deformations. see the dna demo directory for an example of distributed neural network training. === INSTRUCTIONS === --- starting from raw pixels --- * make mnist.test.confusion this will download the mnist training and test sets train a neural network model on mnist from raw pixels evaluate on the original mnist test set and report a confusion matrix results in test errors: 219 out of 10000 disk requirements: about 10 megabytes training time requirements: about 5 minutes on 1 core memory requirements: less than 256 megabytes * make mnisthogwild.test.confusion same as above but "hogwild" training (uses all cores) results in test errors: 203 out of 10000 (typically, but not deterministic) disk requirements: about 10 megabytes training time requirements: about 3 minutes on 4 cores memory requirements: less than 256 megabytes * make mnist8m.test.confusion this will download the mnist test set and the mnist8m training set train a neural network model on mnist8m from raw pixels evaluate on the original mnist test set and report a confusion matrix results in test errors: 152 out of 10000 disk requirements: about 2 gigabytes training time requirements: about an hour on one core memory requirements: about 1.6g (to run infinimist) --- pixel n-grams --- * make mnistpng.test.confusion this will download the mnist training and test sets train a model on mnist linear in pixel n-grams evaluate on the original mnist test set and report a confusion matrix results in test errors: 175 out of 10000 disk requirements: about 10 megabytes training time requirements: about a minute on 3 cores memory requirements: less than 256 megabytes * make mnist11png.test.confusion ibid with 2 neural network hidden units ("goes to eleven") results in test errors: 166 out of 10000 disk requirements: about 10 megabytes training time requirements: about 3 minutes on 3 cores memory requirements: less than 256 megabytes * make mnist8mpng.test.confusion this will download the mnist test set and the mnist8m training set train a model on mnist8m linear in pixel n-grams evaluate on the original mnist test set and report a confusion matrix results in test errors: 140 out of 10000 disk requirements: about 2 gigabytes training time requirements: about an hour on 4 cores memory requirements: less than 256 megabytes * make mnist8m11png.test.confusion ibid with 5 neural network hidden units ("goes to eleven") results in test errors: 107 out of 10000 disk requirements: about 2 gigabytes training time requirements: about two hours on 4 cores memory requirements: less than 256 megabytes vowpal-wabbit-8.5.0.dfsg1/demo/mnist/do-mnist-hogwild-train000077500000000000000000000032351323713302100236070ustar00rootroot00000000000000#! /bin/zsh test -e train-images-idx3-ubyte.gz || { echo "ERROR: you need to download train-image-idx3-ubyte.gz" 1>&2 echo "ERROR: from http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz" 1>&2 exit 1 } test -e train-labels-idx1-ubyte.gz || { echo "ERROR: you need to download train-labels-idx1-ubyte.gz" 1>&2 echo "ERROR: from http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz" 1>&2 exit 1 } set -e nukeem() { \ trap - INT QUIT TERM pkill -9 -f 'vw.*--port 26544' } learner() { netcat localhost 26544 > /dev/null } { ../../vowpalwabbit/vw --oaa 10 -f mnisthogwild.model \ -b 24 --adaptive --invariant --holdout_off \ -l 0.08 --nn 40 \ --daemon --num_children 4 --port 26544 2>&1 | \ perl -lane 'print $_ unless $c{$F[2]}++;' } & trap 'nukeem; exit 1' INT QUIT TERM while ! netcat -z localhost 26544 do sleep 1 done SHUFFLE='BEGIN { srand 69; }; $i = int rand 1000; print $b[$i] if $b[$i]; $b[$i] = $_; } { print grep { defined $_ } @b;' ( for pass in $(seq 1 24) do paste -d' ' \ <(gunzip -c train-labels-idx1-ubyte.gz | ./extract-labels) \ <(gunzip -c train-images-idx3-ubyte.gz | ./extractfeatures) done ) | \ perl -ne ${SHUFFLE} | \ time ./map >(learner) >(learner) >(learner) >(learner) pkill -f 'vw.*--port 26544' while test ! -s mnisthogwild.model do sleep 1 done vowpal-wabbit-8.5.0.dfsg1/demo/mnist/do-mnist-train000077500000000000000000000021321323713302100221470ustar00rootroot00000000000000#! /bin/zsh test -e train-images-idx3-ubyte.gz || { echo "ERROR: you need to download train-image-idx3-ubyte.gz" 1>&2 echo "ERROR: from http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz" 1>&2 exit 1 } test -e train-labels-idx1-ubyte.gz || { echo "ERROR: you need to download train-labels-idx1-ubyte.gz" 1>&2 echo "ERROR: from http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz" 1>&2 exit 1 } SHUFFLE='BEGIN { srand 69; }; $i = int rand 1000; print $b[$i] if $b[$i]; $b[$i] = $_; } { print grep { defined $_ } @b;' paste -d' ' \ <(gunzip -c train-labels-idx1-ubyte.gz | ./extract-labels) \ <(gunzip -c train-images-idx3-ubyte.gz | ./extractfeatures) | \ perl -ne ${SHUFFLE} | \ time ../../vowpalwabbit/vw --oaa 10 -f mnist.model \ -b 24 --adaptive --invariant --holdout_off \ -l 0.1 --nn 40 --passes 24 -k --compressed --cache_file mnist.cache && \ rm -f mnist.cache vowpal-wabbit-8.5.0.dfsg1/demo/mnist/do-mnist11png-train000077500000000000000000000023641323713302100230250ustar00rootroot00000000000000#! /bin/zsh test -e train-images-idx3-ubyte.gz || { echo "ERROR: you need to download train-image-idx3-ubyte.gz" 1>&2 echo "ERROR: from http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz" 1>&2 exit 1 } test -e train-labels-idx1-ubyte.gz || { echo "ERROR: you need to download train-labels-idx1-ubyte.gz" 1>&2 echo "ERROR: from http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz" 1>&2 exit 1 } SHUFFLE='BEGIN { srand 69; }; $i = int rand 1000; print $b[$i] if $b[$i]; $b[$i] = $_; } { print grep { defined $_ } @b;' paste -d' ' \ <(gunzip -c train-labels-idx1-ubyte.gz | ./extract-labels) \ <(gunzip -c train-images-idx3-ubyte.gz | ./extractpixels) | \ perl -ne ${SHUFFLE} | \ ./roundrobin ./pixelngrams 2 | \ time ../../vowpalwabbit/vw --oaa 10 -f mnist11png.model \ -b 20 --adaptive --invariant \ --nn 2 --inpass --holdout_off \ -l 0.02 --passes 15 -k --compressed --cache_file mnist11png.cache && \ rm -f mnist11png.cache vowpal-wabbit-8.5.0.dfsg1/demo/mnist/do-mnist8m-train000077500000000000000000000015341323713302100224210ustar00rootroot00000000000000#! /bin/zsh test -e train8m-images-idx3-ubyte.gz || { echo "ERROR: you need to download train8m-image-idx3-ubyte.gz" 1>&2 echo "ERROR: from http://ml.nec-labs.com/download/data/mnist8m/train8m-images-idx3-ubyte.gz" 1>&2 exit 1 } test -e train8m-labels-idx1-ubyte.gz || { echo "ERROR: you need to download train8m-labels-idx1-ubyte.gz" 1>&2 echo "ERROR: from http://ml.nec-labs.com/download/data/mnist8m/train8m-labels-idx1-ubyte.gz" 1>&2 exit 1 } paste -d' ' \ <(gunzip -dc train8m-labels-idx1-ubyte.gz | ./extract-labels) \ <(gunzip -dc train8m-images-idx3-ubyte.gz | ./extractfeatures) | \ time ../../vowpalwabbit/vw --oaa 10 -f mnist8m.model \ -b 24 --adaptive --invariant --holdout_off \ -l 0.1 --nn 40 vowpal-wabbit-8.5.0.dfsg1/demo/mnist/do-mnist8m11png-train000077500000000000000000000022771323713302100232750ustar00rootroot00000000000000#! /bin/zsh test -e train8m-images-idx3-ubyte.gz || { echo "ERROR: you need to download train8m-image-idx3-ubyte.gz" 1>&2 echo "ERROR: from http://ml.nec-labs.com/download/data/mnist8m/train8m-images-idx3-ubyte.gz" 1>&2 exit 1 } test -e train8m-labels-idx1-ubyte.gz || { echo "ERROR: you need to download train8m-labels-idx1-ubyte.gz" 1>&2 echo "ERROR: from http://ml.nec-labs.com/download/data/mnist8m/train8m-labels-idx1-ubyte.gz" 1>&2 exit 1 } SHUFFLE='BEGIN { srand 69; }; $i = int rand 1000; print $b[$i] if $b[$i]; $b[$i] = $_; } { print grep { defined $_ } @b;' paste -d' ' \ <(gunzip -c train8m-labels-idx1-ubyte.gz | ./extract-labels) \ <(gunzip -c train8m-images-idx3-ubyte.gz | ./extractpixels) | \ perl -ne ${SHUFFLE} | \ ./roundrobin ./pixelngrams 3 | \ time ../../vowpalwabbit/vw --oaa 10 -f mnist8m11png.model \ -b 20 --adaptive --invariant --holdout_off \ --nn 5 --inpass \ -l 0.05 vowpal-wabbit-8.5.0.dfsg1/demo/mnist/do-mnist8mpng-train000077500000000000000000000021651323713302100231270ustar00rootroot00000000000000#! /bin/zsh test -e train8m-images-idx3-ubyte.gz || { echo "ERROR: you need to download train8m-image-idx3-ubyte.gz" 1>&2 echo "ERROR: from http://ml.nec-labs.com/download/data/mnist8m/train8m-images-idx3-ubyte.gz" 1>&2 exit 1 } test -e train8m-labels-idx1-ubyte.gz || { echo "ERROR: you need to download train8m-labels-idx1-ubyte.gz" 1>&2 echo "ERROR: from http://ml.nec-labs.com/download/data/mnist8m/train8m-labels-idx1-ubyte.gz" 1>&2 exit 1 } SHUFFLE='BEGIN { srand 69; }; $i = int rand 1000; print $b[$i] if $b[$i]; $b[$i] = $_; } { print grep { defined $_ } @b;' paste -d' ' \ <(gunzip -c train8m-labels-idx1-ubyte.gz | ./extract-labels) \ <(gunzip -c train8m-images-idx3-ubyte.gz | ./extractpixels) | \ perl -ne ${SHUFFLE} | \ ./roundrobin ./pixelngrams 3 | \ time ../../vowpalwabbit/vw --oaa 10 -f mnist8mpng.model \ -b 20 --adaptive --invariant --holdout_off \ -l 0.05 vowpal-wabbit-8.5.0.dfsg1/demo/mnist/do-mnistpng-train000077500000000000000000000022501323713302100226550ustar00rootroot00000000000000#! /bin/zsh test -e train-images-idx3-ubyte.gz || { echo "ERROR: you need to download train-image-idx3-ubyte.gz" 1>&2 echo "ERROR: from http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz" 1>&2 exit 1 } test -e train-labels-idx1-ubyte.gz || { echo "ERROR: you need to download train-labels-idx1-ubyte.gz" 1>&2 echo "ERROR: from http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz" 1>&2 exit 1 } SHUFFLE='BEGIN { srand 69; }; $i = int rand 1000; print $b[$i] if $b[$i]; $b[$i] = $_; } { print grep { defined $_ } @b;' paste -d' ' \ <(gunzip -c train-labels-idx1-ubyte.gz | ./extract-labels) \ <(gunzip -c train-images-idx3-ubyte.gz | ./extractpixels) | \ perl -ne ${SHUFFLE} | \ ./roundrobin ./pixelngrams 2 | \ time ../../vowpalwabbit/vw --oaa 10 -f mnistpng.model \ -b 20 --adaptive --invariant --holdout_off \ -l 0.05 --passes 15 -k --compressed --cache_file mnistpng.cache && \ rm -f mnistpng.cache vowpal-wabbit-8.5.0.dfsg1/demo/mnist/do-pixelngram-test000077500000000000000000000015631323713302100230340ustar00rootroot00000000000000#! /bin/zsh test -e $1.model || { echo "ERROR: you need to train a model" 1>&2 echo "ERROR: run do-${1}-train in this directory" 1>&2 exit 1 } test -e t10k-labels-idx1-ubyte.gz || { echo "ERROR: you need to download t10k-labels-idx1-ubyte.gz" 1>&2 echo "ERROR: from http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz" 1>&2 exit 1 } test -e t10k-images-idx3-ubyte.gz || { echo "ERROR: you need to download t10k-images-idx3-ubyte.gz" 1>&2 echo "ERROR: from http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz" 1>&2 exit 1 } paste -d' ' \ <(gunzip -c t10k-labels-idx1-ubyte.gz | ./extract-labels) \ <(gunzip -c t10k-images-idx3-ubyte.gz | \ ./extractpixels | ./pixelngrams) | \ ../../vowpalwabbit/vw -t -i $1.model -p $1.test.predictions vowpal-wabbit-8.5.0.dfsg1/demo/mnist/do-test000077500000000000000000000014731323713302100206700ustar00rootroot00000000000000#! /bin/zsh test -e $1.model || { echo "ERROR: you need to train a model" 1>&2 echo "ERROR: run do-${1}-train in this directory" 1>&2 exit 1 } test -e t10k-labels-idx1-ubyte.gz || { echo "ERROR: you need to download t10k-labels-idx1-ubyte.gz" 1>&2 echo "ERROR: from http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz" 1>&2 exit 1 } test -e t10k-images-idx3-ubyte.gz || { echo "ERROR: you need to download t10k-images-idx3-ubyte.gz" 1>&2 echo "ERROR: from http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz" 1>&2 exit 1 } paste -d' ' \ <(gunzip -c t10k-labels-idx1-ubyte.gz | ./extract-labels) \ <(gunzip -c t10k-images-idx3-ubyte.gz | ./extractfeatures) | \ ../../vowpalwabbit/vw -t -i $1.model -p $1.test.predictions vowpal-wabbit-8.5.0.dfsg1/demo/mnist/extract-labels000077500000000000000000000010571323713302100222210ustar00rootroot00000000000000#! /usr/bin/env perl use warnings; use strict; use IO::File; local $SIG{__WARN__} = sub {}; #my $fname = shift @ARGV or die; #my $fh = new IO::File $fname, "r" or die "$fname: $!"; my $fh = \*STDIN; binmode ($fh, ':raw'); $/ = \4; my $magic = unpack ("N", <$fh>); die "wtf $magic" unless $magic == 2049; my $n_items = unpack ("N", <$fh>); warn "n_items = $n_items"; $/ = \1; while (defined ($_ = <$fh>)) { die "wtf @{[length($_)]}" unless length ($_) == 1; my ($label) = unpack ("C", $_); ++$label; print "$label 1 $label|\n"; } vowpal-wabbit-8.5.0.dfsg1/demo/mnist/extractfeatures.cpp000066400000000000000000000021601323713302100232720ustar00rootroot00000000000000#include #include #include #include #include #include int main (void) { using std::cin; using std::cout; using std::endl; using std::setprecision; uint32_t magic; cin.read (reinterpret_cast (&magic), sizeof (uint32_t)); magic = ntohl (magic); assert (magic == 2051); uint32_t n_images; cin.read (reinterpret_cast (&n_images), sizeof (uint32_t)); n_images = ntohl (n_images); uint32_t n_rows; cin.read (reinterpret_cast (&n_rows), sizeof (uint32_t)); n_rows = ntohl (n_rows); uint32_t n_columns; cin.read (reinterpret_cast (&n_columns), sizeof (uint32_t)); n_columns = ntohl (n_columns); uint32_t rc = n_rows * n_columns; unsigned char buf[rc]; for (cin.read (reinterpret_cast (buf), rc); ! cin.eof (); cin.read (reinterpret_cast (buf), rc)) { cout << "|p"; for (unsigned int p = 0; p < n_rows * n_columns; ++p) { if (buf[p]) cout << " " << p << ":" << setprecision (8) << static_cast(buf[p])/256.0; } cout << endl; } return 0; } vowpal-wabbit-8.5.0.dfsg1/demo/mnist/extractpixels.cpp000066400000000000000000000021111323713302100227540ustar00rootroot00000000000000#include #include #include #include #include #include int main (void) { using std::cin; using std::cout; using std::endl; using std::setprecision; uint32_t magic; cin.read (reinterpret_cast (&magic), sizeof (uint32_t)); magic = ntohl (magic); assert (magic == 2051); uint32_t n_images; cin.read (reinterpret_cast (&n_images), sizeof (uint32_t)); n_images = ntohl (n_images); uint32_t n_rows; cin.read (reinterpret_cast (&n_rows), sizeof (uint32_t)); n_rows = ntohl (n_rows); uint32_t n_columns; cin.read (reinterpret_cast (&n_columns), sizeof (uint32_t)); n_columns = ntohl (n_columns); uint32_t rc = n_rows * n_columns; unsigned char buf[rc]; for (cin.read (reinterpret_cast (buf), rc); ! cin.eof (); cin.read (reinterpret_cast (buf), rc)) { for (unsigned int p = 0; p < n_rows * n_columns; ++p) { if (buf[p]) cout << " " << p << ":" << static_cast(buf[p]); } cout << endl; } return 0; } vowpal-wabbit-8.5.0.dfsg1/demo/mnist/map000077500000000000000000000004371323713302100200650ustar00rootroot00000000000000#! /usr/bin/env perl use warnings; use strict; use IO::File; my @mappers = map { $_->autoflush (0); $_ } map { new IO::File $_, "w" or die "$_: $!" } @ARGV; while (defined ($_ = )) { my $fh = $mappers[$. % @mappers]; print $fh $_; } vowpal-wabbit-8.5.0.dfsg1/demo/mnist/pixelngrams.cpp000066400000000000000000000065441323713302100224240ustar00rootroot00000000000000#include #include #include #include #include #include #include namespace { //====================================================================== // vhd_ngrams = // = // vertical, horizontal, and diagonal 2-skip-2 grams. = //======================================================================/ void vhd_ngrams (unsigned char*const buf, size_t p, size_t stride, size_t total, double* v) { double x = static_cast (buf[p]) / 256.0; double xp1 = static_cast (buf[(p + 1) % total]) / 256.0; double xp2 = static_cast (buf[(p + 2) % total]) / 256.0; double xp3 = static_cast (buf[(p + 3) % total]) / 256.0; double xd1 = static_cast (buf[(p + stride) % total]) / 256.0; double xd2 = static_cast (buf[(p + 2 * stride) % total]) / 256.0; double xd3 = static_cast (buf[(p + 3 * stride) % total]) / 256.0; double xpd1 = static_cast (buf[(p + stride + 1) % total]) / 256.0; double xpd2 = static_cast (buf[(p + 2 * (stride + 1)) % total]) / 256.0; double xpd3 = static_cast (buf[(p + 3 * (stride + 1)) % total]) / 256.0; double xmd1 = static_cast (buf[(p + stride - 1) % total]) / 256.0; double xmd2 = static_cast (buf[(p + 2 * (stride - 1)) % total]) / 256.0; double xmd3 = static_cast (buf[(p + 3 * (stride - 1)) % total]) / 256.0; v[0] = ::sqrt (x * xp1); v[1] = ::sqrt (x * xd1); v[2] = ::sqrt (x * xpd1); v[3] = ::sqrt (x * xmd1); v[4] = ::sqrt (x * xp2); v[5] = ::sqrt (x * xd2); v[6] = ::sqrt (x * xpd2); v[7] = ::sqrt (x * xmd2); v[8] = ::sqrt (x * xp3); v[9] = ::sqrt (x * xd3); v[10] = ::sqrt (x * xpd3); v[11] = ::sqrt (x * xmd3); } } int main (void) { using std::cin; using std::cout; using std::getline; using std::endl; using std::flush; using std::string; size_t n_rows = 28; size_t n_columns = 28; size_t rc = n_rows * n_columns; unsigned char buf[n_rows * n_columns]; while (! cin.eof ()) { string line; getline (cin, line); if (line.length ()) { unsigned int feature; unsigned int value; size_t start = 0; int increment; char* pipe = const_cast (strchr (line.c_str (), '|')); if (pipe) { *pipe++ = '\0'; cout << line.c_str (); start = pipe - line.c_str (); } ::memset (buf, 0, n_rows * n_columns * sizeof (buf[0])); while (::sscanf (line.c_str () + start, "%u:%u%n", &feature, &value, &increment) >= 2) { buf[feature] = value; start += increment; } size_t offset = 1; cout << "|p"; for (unsigned int p = 0; p < n_rows * n_columns; ++p) { if (buf[p]) { cout << " " << offset << ":" << static_cast(buf[p])/256.0; double v[12]; vhd_ngrams (buf, p, n_rows, rc, v); for (unsigned int j = 0; j < 12; ++j) { if (v[j]) { cout << " " << offset + 1 + j << ":" << v[j]; } } } offset += 13; } cout << endl << flush; } } return 0; } vowpal-wabbit-8.5.0.dfsg1/demo/mnist/roundrobin000077500000000000000000000033021323713302100214630ustar00rootroot00000000000000#! /usr/bin/env perl use warnings; use strict; use IO::File; use IO::Pipe; use IO::Select; sub setup_child ($$$) { my ($command, $n, $t) = @_; my $pipec2p = new IO::Pipe; my $pipep2c = new IO::Pipe; my $pid = fork (); if ($pid) # parent { $pipec2p->reader (); $pipep2c->writer (); return [ $pipec2p, $pipep2c ]; } else # child { $pipec2p->writer (); $pipep2c->reader (); my $writefno = $pipec2p->fileno (); my $readfno = $pipep2c->fileno (); open STDIN, "<&$readfno" or die "can't dup $readfno to STDIN"; open STDOUT, ">&$writefno" or die "can't dup $writefno to STDOUT"; exec $command, $n, $t or die "can't exec $command"; } } my $command = shift @ARGV or die; my $total = shift @ARGV or die; my @mappers = map { setup_child ($command, $_, $total) } 1 .. $total; my @read = map { my $s = new IO::Select (); $s->add ($_->[0]); $s } @mappers; my @write = map { my $s = new IO::Select (); $s->add ($_->[1]); $s } @mappers; my $in = 0; my $out = 0; while (defined ($_ = )) { while (! $write[$out % @write]->can_write (0.001)) { while ($read[$in % @read]->can_read (0)) { my $fh = $mappers[$in % @mappers]->[0]; print $fh->getline (); ++$in; } } my $fh = $mappers[$out % @write]->[1]; $fh->printflush ($_); ++$out; } foreach my $m (@mappers) { my $fh = $m->[1]; $fh->close (); } while ($in < $out) { my $fh = $mappers[$in % @mappers]->[0]; print $fh->getline (); ++$in; } vowpal-wabbit-8.5.0.dfsg1/demo/movielens/000077500000000000000000000000001323713302100202255ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/demo/movielens/.gitignore000066400000000000000000000001531323713302100222140ustar00rootroot00000000000000*.results *.results.model *.results.model.txt ml-1m ml-1m.ratings.train.vw ml-1m.ratings.test.vw ml-1m.zip vowpal-wabbit-8.5.0.dfsg1/demo/movielens/Makefile000066400000000000000000000150261323713302100216710ustar00rootroot00000000000000SHELL=/bin/zsh VW=../../vowpalwabbit/vw .SECONDARY: all: @cat README.md shootout: $(foreach what,linear lrq lrqdropout lrqdropouthogwild,$(what).print) clean: rm -f $(wildcard *results*) $(wildcard *.vw) $(wildcard *.model.txt) $(wildcard *.pdf) ml-%.zip: @echo "downloading movielens $*" @wget http://files.grouplens.org/datasets/movielens/ml-$*.zip ml-%/ratings.dat: ml-%.zip @rm -rf ml-$* @unzip -qq $< @(test -d ml-10M100K && mv -f ml-10M100K ml-10m) || true @rm -rf __MACOSX @touch ml-$*/* ml-%.ratings.train.vw: ml-%/ratings.dat @echo -n "preprocessing movielens $* ..." @./ratings2vw ml-$*.ratings.pre.train.vw ml-$*.ratings.test.vw $< @perl -ne 'BEGIN { srand 8675309; }; \ 1; print join "\t", rand (), $$_;' \ ml-$*.ratings.pre.train.vw | sort -k1 | \ cut -f2- > ml-$*.ratings.train.vw @rm -f ml-$*.ratings.pre.train.vw @echo " complete" %.test.vw: %.train.vw @true %.print: %.results @printf "%s test MAE is %3.3f\n" $* $$(cat $*.results) #--------------------------------------------------------------------- # linear model (no interaction terms) #--------------------------------------------------------------------- linear.results: ml-1m.ratings.test.vw ml-1m.ratings.train.vw @echo "****************************************************" @echo "* training linear model (no interaction terms) *" @echo "****************************************************" @echo @${VW} --loss_function quantile -l 1 -b 24 --passes 100 \ -k --cache_file $@.cache -d $(word 2,$+) --holdout_off \ --adaptive --invariant -f $@.model @echo "****************************************************" @echo "* testing linear model (no interaction terms) *" @echo "****************************************************" @echo @${VW} --loss_function quantile -t -i $@.model \ -d $(word 1,$+) -p \ >(perl -lane '$$s+=abs(($$F[0]-$$F[1])); } { \ 1; print $$s/$$.;' > $@) @echo "****************************************************" @echo "* saving human readable model (--invert_hash) *" @echo "****************************************************" @echo @${VW} -i $@.model -t --invert_hash $@.model.txt -d $(word 2,$+) @rm -f $@.cache $@.model #--------------------------------------------------------------------- # low-rank interaction model (without dropout) #--------------------------------------------------------------------- lrq.results: ml-1m.ratings.test.vw ml-1m.ratings.train.vw @echo "*********************************************************" @echo "* training low-rank interaction model (without dropout) *" @echo "* *" @echo "* vw --lrq um7 ... *" @echo "*********************************************************" @echo @${VW} --loss_function quantile -l 0.1 -b 24 --passes 100 \ -k --cache_file $@.cache -d $(word 2,$+) --holdout_off \ --power_t 0.333 --l2 1.25e-7 --lrq um7 --adaptive --invariant -f $@.model @echo "********************************************************" @echo "* testing low-rank interaction model (without dropout) *" @echo "********************************************************" @echo @${VW} --loss_function quantile -t -i $@.model \ -d $(word 1,$+) -p \ >(perl -lane '$$s+=abs(($$F[0]-$$F[1])); } { \ 1; print $$s/$$.;' > $@) @echo "****************************************************" @echo "* saving human readable model (--invert_hash) *" @echo "****************************************************" @echo @${VW} -i $@.model -t --invert_hash $@.model.txt -d $(word 2,$+) @rm -f $@.cache $@.model #--------------------------------------------------------------------- # low-rank interaction model (with dropout) #--------------------------------------------------------------------- lrqdropout.results: ml-1m.ratings.test.vw ml-1m.ratings.train.vw @echo "******************************************************" @echo "* training low-rank interaction model (with dropout) *" @echo "* *" @echo "* vw --lrq um14 --lrqdropout ... *" @echo "******************************************************" @echo @${VW} --loss_function quantile -l 0.45 -b 24 --passes 100 \ -k --cache_file $@.cache -d $(word 2,$+) --holdout_off \ --lrq um14 --lrqdropout --adaptive --invariant -f $@.model @echo "*****************************************************" @echo "* testing low-rank interaction model (with dropout) *" @echo "*****************************************************" @echo @${VW} --loss_function quantile -t -i $@.model \ -d $(word 1,$+) -p \ >(perl -lane '$$s+=abs(($$F[0]-$$F[1])); } { \ 1; print $$s/$$.;' > $@) @echo "****************************************************" @echo "* saving human readable model (--invert_hash) *" @echo "****************************************************" @echo @${VW} -i $@.model -t --invert_hash $@.model.txt -d $(word 2,$+) @rm -f $@.cache $@.model movie_dendrogram.pdf: lrqdropout.results @echo "*******************************************************" @echo "* Generating movie clustering based on latent factors *" @echo "* (Requires an installation of R) *" @echo "*******************************************************" @grep ^lrq^m $<.model.txt | tr '^' ':' > $<.model.csv @Rscript visualize_factors.R @rm -f $@.model.csv #--------------------------------------------------------------------- # low-rank interaction model (with dropout) # (HOGWILD training mode) #--------------------------------------------------------------------- lrqdropouthogwild.results: ml-1m.ratings.test.vw ml-1m.ratings.train.vw do-lrq-hogwild @echo "******************************************************" @echo "* training low-rank interaction model (with dropout) *" @echo "* (HOGWILD training mode) *" @echo "* *" @echo "* vw --lrq um14 --lrqdropout ... *" @echo "******************************************************" @echo @./do-lrq-hogwild $@.model @echo "*****************************************************" @echo "* testing low-rank interaction model (with dropout) *" @echo "*****************************************************" @echo @${VW} --loss_function quantile -t -i $@.model \ -d $(word 1,$+) -p \ >(perl -lane '$$s+=abs(($$F[0]-$$F[1])); } { \ 1; print $$s/$$.;' > $@) .PHONY: all clean shootout vowpal-wabbit-8.5.0.dfsg1/demo/movielens/README.md000077500000000000000000000073331323713302100215150ustar00rootroot00000000000000Low rank quadratic demo ------------------------------- This demo shows a low-rank approximation to an interaction design matrix for the [movielens-1M](http://files.grouplens.org/papers/ml-10m-README.html) dataset. ### About low-rank interactions ### In movielens-1M, a user has at most one rating per movie, and therefore a full interaction design between these two variables (in `vw` syntax: `-q um`) fundamentally cannot generalize. Since this situation arises in recommendation systems, low-rank approximations to interaction terms rose to prominence in the recommendation community, under the moniker "matrix factorization". However, the technique is also appropriate in non-recommendation settings, e.g., when the interaction between two high cardinality categorical variables is desired but the available data is too sparse to learn a full interaction model. There is a great piece of software called [libfm](http://www.libfm.org/) whose raison d'etre is to fit low-rank approximations to interaction designs, and the main author [Steffen Rendle](http://www.kaggle.com/users/25112/steffen-rendle) does quite well on Kaggle. Imitation is the best form of flattery. ### How it works ### If you have two namespaces `a` and `b`, instead of the full interaction design enabled by specifying `-q ab`, you can have a rank-k interaction design by specifying `--lrq abk`. Additionally specifying `--lrqdropout` trains with dropout which sometimes works better. When using dropout the best performing rank tends to be about twice as big as without dropout. You might find a bit of `--l2` regularization improves generalization. ### Demo Instructions ### - `make shootout`: eventually produces four results indicating test MAE (mean absolute error) on movielens-1M for - linear: a model without any interactions. basically this creates a user bias and item bias fit. this is a surprisingly strong baseline in terms of MAE, but is useless for recommendation as it induces the same item ranking for all users. It achieves test MAE of 0.731. - lrq: the linear model augmented with rank-7 interactions between users and movies, aka, "seven latent factors". It achieves test MAE of 0.709. I determined that 7 was the best number to use through experimentation. The key additional `vw` command-line flags vs. the linear model are `--l2 1.25e-7 --lrq um7`. Performance is sensitive to the choice of `--l2` regularization strength. - lrqdropout: the linear model augmented with rank-14 interactions between users and movies, and trained with dropout. It achieves test MAE of 0.689. The key additional `vw` command-line flags vs. the linear model are `--lrq um14 --lrqdropout`. - lrqdropouthogwild: same as lrqdropout, but trained in parallel on multiple cores without locking, a la [Niu et. al.](http://www.eecs.berkeley.edu/~brecht/papers/hogwildTR.pdf). Test MAE is nondeterministic but typically equivalent to lrqdropout. The main purpose of this demo is to instruct on how to achieve lock-free parallel learning. (Note using the cache and a single training core can be faster than using multiple cores and parsing continuously. However in some cases data is generated dynamically in such volume that the cache is not practical, thus this technique is helpful.) - the first time you invoke `make shootout` there is a lot of other output. invoking it a second time will allow you to just see the cached results. - `make movie_dendrogram.pdf` will produce a couple of PDFs with hierarchical clustering of the movies based on the latent factors found by `--lrq`. It serves as an example on how to extract the latent factors from an `--invert_hash` file. You will need to zoom in in the large dendrogram to find the movie names. Details about how `vw` is invoked is in the `Makefile`. vowpal-wabbit-8.5.0.dfsg1/demo/movielens/do-lrq-hogwild000077500000000000000000000020111323713302100227760ustar00rootroot00000000000000#! /bin/zsh nukeem() { \ trap - INT QUIT TERM pkill -9 -f 'vw.*--port 26542' } if which netcat 2>/dev/null; then netcat=netcat elif which nc 2>/dev/null; then netcat=nc else echo "Install netcat or nc" >&2 exit 1 fi rm -f "$1" { ../../vowpalwabbit/vw --loss_function quantile -l 0.45 -b 24 --holdout_off --lrq um14 --lrqdropout --adaptive --invariant -f "$1" --daemon --num_children 4 --port 26542 2>&1 | perl -lane 'print $_ unless $c{$F[2]}++; ' } & trap 'nukeem; exit 1' INT QUIT TERM while ! $netcat -z localhost 26542 do sleep 1 done for pass in $(seq 1 100) do cat ml-1m.ratings.train.vw done | \ ./map \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) pkill -f 'vw.*--port 26542' while test ! -s "$1" do sleep 1 done vowpal-wabbit-8.5.0.dfsg1/demo/movielens/map000077500000000000000000000006411323713302100207310ustar00rootroot00000000000000#! /usr/bin/env perl use warnings; use strict; use IO::File; my @mappers = map { $_->autoflush (0); $_ } map { new IO::File $_, "w" or die "$_: $!" } @ARGV; my @buf = grep { defined } map { scalar } (0 .. 49); my $n = 0; while (@buf) { my $fh = $mappers[$n % @mappers]; print $fh @buf; ++$n; @buf = grep { defined } map { scalar } (0 .. 49); } vowpal-wabbit-8.5.0.dfsg1/demo/movielens/ratings2vw000077500000000000000000000024171323713302100222650ustar00rootroot00000000000000#! /usr/bin/env perl use IO::File; use warnings; use strict; srand 69; sub output_user ($$@) { my ($trainfh, $testfh, @rows) = @_; return () unless @rows > 1; my @permrows = map { $_->[1] } sort { $a->[0] <=> $b->[0] } map { [ rand (), $_ ] } @rows; my @testrows = splice @permrows, -1; print $trainfh @permrows; return @testrows; } my $trainfile = shift @ARGV or die; my $testfile = shift @ARGV or die; my $trainfh = new IO::File $trainfile, "w" or die; my $testfh = new IO::File $testfile, "w" or die; my $olduser; my @rows; my @save; my %seen; while (defined ($_ = <>)) { chomp; my ($user, $movie, $rating, undef) = split /::/, $_; if (defined ($olduser) && $user != $olduser) { push @save, output_user ($trainfh, $testfh, @rows); undef @rows; die "input file not collated" if $seen{$olduser}++; } push @rows, "$rating $rating|user $user |movie $movie\n"; $olduser = $user; } push @save, output_user ($trainfh, $testfh, @rows); my @permsave = map { $_->[1] } sort { $a->[0] <=> $b->[0] } map { [ rand (), $_ ] } @save; my @test = splice @permsave, 0, 5000; print $trainfh @permsave; print $testfh @test; vowpal-wabbit-8.5.0.dfsg1/demo/movielens/visualize_factors.R000066400000000000000000000045661323713302100241170ustar00rootroot00000000000000# Create movie dendrogram based on latent factors library(reshape) ########################################################## # Utility functions ########################################################## # Load factors file (as preprocessed by shell script) and join with movie names # for better visualization loadMovieFactors <- function(name) { lrq_movies <- read.csv('lrqdropout.results.model.csv',header=F,sep=':') names(lrq_movies) <- c('lrq','namespace','movie','factor','hash','weight') movie_list <- read.csv('ml-1m/movies.dat',header=F,sep=':',fileEncoding='latin1') movies <- data.frame(movie=movie_list$V1,name=movie_list$V3,genre=movie_list$V5) movies$full_name <- paste(movies$name,movies$genre,sep=' / ') mm <- merge(lrq_movies,movies,by='movie') movie_factors <- cast(mm, full_name ~ factor, value='weight', sum) rownames(movie_factors) <- movie_factors$full_name subset(movie_factors, select = -c(full_name)) } # Calculate distances between movie latent factors using cosine similarity cosineSimilarity <- function(df){ x <- as.matrix(df) m <- 1 - x%*%t(x)/(sqrt(rowSums(x^2) %*% t(rowSums(x^2)))) rownames(m) <- rownames(df) colnames(m) <- rownames(df) m } # Generate a hierarchical clustering dendrogram, color the N cluster founds with alternating colors # The movie titles are going to be really small because of the number of movies, better to use # a dendrogram viewing tool. clusterMovies <- function(distances, N) { hc <- hclust(distances) clusMember <- cutree(hc,N) labelColors <- rep(c("#036564", "#EB6841"), N/2) colLab <- function(n) { if (is.leaf(n)) { a <- attributes(n) labCol <- labelColors[clusMember[which(names(clusMember) == a$label)]] attr(n, "nodePar") <- c(a$nodePar, lab.col = labCol) } n } hcd <- as.dendrogram(hc) dendrapply(hcd, colLab) } # Save the dendrogram in PDF so that it is possible to zoom into the movie names saveDendrogram <- function(dendrogram,fname,w=40,h=15,cex=1.0) { pdf(fname, width=w, height=h) par(cex=cex,mai=c(8,2,2,2)) plot(dendrogram) dev.off() } movie_factors <- loadMovieFactors('lrqdropout.results.model.csv') distances <- cosineSimilarity(movie_factors) dendrogram <- clusterMovies(as.dist(distances), 80) saveDendrogram(dendrogram,"movie_dendrogram.pdf",w=400,h=150,cex=0.5) saveDendrogram(cut(dendrogram,h=1.5)$lower[[1]],"movie_dendrogram_small.pdf",w=20) vowpal-wabbit-8.5.0.dfsg1/demo/normalized/000077500000000000000000000000001323713302100203705ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/demo/normalized/.gitignore000066400000000000000000000000511323713302100223540ustar00rootroot00000000000000*.gz *.zip *.Z *.bz2 *.best *.nonormbest vowpal-wabbit-8.5.0.dfsg1/demo/normalized/Makefile000066400000000000000000000256331323713302100220410ustar00rootroot00000000000000SHELL=/bin/bash VW=../../vowpalwabbit/vw .SECONDARY: all: @cat README.md datasets := bank census covertype CTslice MSD shuttle all.results.pre: $(foreach what,$(datasets),$(what).best $(what).nonormbest) @printf "%20.20s\t%9s\t%9s\t%9s\t%9s\n" "Dataset" "NAG eta^*" "NAG loss" "AG eta^*" "AG loss" all.results: all.results.pre $(foreach what,$(datasets),$(what).resultsprint) SHUFFLE='BEGIN { srand 69; }; \ $$i = int rand 100000; \ print $$b[$$i] if $$b[$$i]; \ $$b[$$i] = $$_; } { print grep { defined $$_ } @b;' #--------------------------------------------------------------------- # bank marketing # # normalization really helps. The columns have units of euros, seconds, # days, and years; in addition there are categorical variables. #--------------------------------------------------------------------- bank.zip: @echo "downloading bank ..." 1>&2 @wget -q -O - \ http://archive.ics.uci.edu/ml/machine-learning-databases/00222/$@ \ > $@ bank.preprocess.data.gz: bank.zip @echo "preprocessing bank ..." 1>&2 @zcat $< 2>/dev/null | \ perl -MScalar::Util -F';' -lane '$$l = pop @F; $$p = ($$l =~ /no/) ? -1 : 1; print "$$p $$p|f ", join " ", map { $$isn = Scalar::Util::looks_like_number ($$F[$$_]); $$F[$$_] =~ s/^\s+//; $$F[$$_] =~ s/\s+$$//; $$F[$$_] =~ s/\W/_/g unless $$isn; $$isn ? "@{[$$_+1]}:$$F[$$_]" : "@{[$$_+1]}_$$F[$$_]" } grep { $$F[$$_] =~ /\w/ && ( ! Scalar::Util::looks_like_number ($$F[$$_]) || $$F[$$_] > 0 ) } (0 .. $$#F)' | \ perl -e 'BEGIN { srand 69; }; print map { $$_->[1] } sort { $$a->[0] <=> $$b->[0] } map { [ rand (), $$_ ] } <>;' | gzip \ > $@ bank.data: bank.preprocess.data.gz bank.%.nonormlearn: bank.preprocess.data.gz @${VW} --loss_function logistic -b 22 -q ff -l $* $< \ --adaptive --invariant \ -p >(perl -lane ' \ 1; ++$$n; $$l+=1.0 if $$F[0] * $$F[1] < 0; } \ 1; { printf "average loss = %f\t%u\t%u\n", $$l/$$n, $$l, $$n;') bank.%.learn: bank.preprocess.data.gz @${VW} --loss_function logistic -b 22 -q ff -l $* $< \ -p >(perl -lane ' \ 1; ++$$n; $$l+=1.0 if $$F[0] * $$F[1] < 0; } \ 1; { printf "average loss = %f\t%u\t%u\n", $$l/$$n, $$l, $$n;') bankbestmin=1e-2 bankbestmax=10 banknonormbestmin=1e-8 banknonormbestmax=1e-3 banktimeestimate=1 #--------------------------------------------------------------------- # covertype #--------------------------------------------------------------------- covtype.data.gz: @echo "downloading covertype ..." @wget -q -O - \ http://archive.ics.uci.edu/ml/machine-learning-databases/covtype/$@ \ > $@ covtype.preprocess.data.gz: covtype.data.gz @echo "preprocessing covertype ..." 1>&2 @perl -F',' \ -lane '$$l = pop @F; print "$$l $$l|f ", map { "$$_:$$F[$$_] "} grep { $$F[$$_] } (0 .. $$#F)' <(zcat $<) | \ perl -e 'BEGIN { srand 69; }; print map { $$_->[1] } sort { $$a->[0] <=> $$b->[0] } map { [ rand (), $$_ ] } <>;' | gzip \ > $@ covertype.data: covtype.preprocess.data.gz covertype.%.nonormlearn: covtype.preprocess.data.gz ${VW} --adaptive --invariant -b 22 --hash all -q ff --oaa 7 -l $* -d $< covertype.%.learn: covtype.preprocess.data.gz ${VW} -b 22 --hash all -q ff --oaa 7 -l $* -d $< covertypebestmin=1e-2 covertypebestmax=10 covertypenonormbestmin=1e-8 covertypenonormbestmax=1e-5 covertypetimeestimate=5 #--------------------------------------------------------------------- # million song database # # normalization is helpful. #--------------------------------------------------------------------- YearPredictionMSD.bz2: @echo "downloading MSD ..." 1>&2 @wget -q -O - \ http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/regression/$@ \ > $@ year.preprocess.data.gz: YearPredictionMSD.bz2 @echo "preprocessing MSD ..." 1>&2 @perl -lane 'splice @F, 1, 0, "$$F[0]|f"; print join " ", @F' \ <(bzcat $<) | \ perl -ne $(SHUFFLE) \ | gzip > $@ MSD.data: year.preprocess.data.gz MSD.%.learn: year.preprocess.data.gz @zcat $< | \ perl -lane '$$F[0] -= 1998; print join " ", @F' | \ ${VW} -q ff -l $* \ -p >(perl -lane ' \ 1; BEGIN { $$max = 0; $$min = 3000; }; \ 1; ++$$n; $$l += ($$F[0] + 1998 - $$F[1])**2; \ 1; $$max = $$F[1] if $$max < $$F[1]; \ 1; $$min = $$F[1] if $$min > $$F[1]; \ 1; } { $$c = $$l / ($$max - $$min)**2; \ 1; printf "average loss = %f\t%.3f\t%.3f\t%.3f\t%.3f\n", $$c/$$n, $$l, $$n, $$min, $$max;') MSD.%.nonormlearn: year.preprocess.data.gz @zcat $< | \ perl -lane '$$F[0] -= 1998; print join " ", @F' | \ ${VW} --adaptive --invariant -q ff -l $* \ -p >(perl -lane ' \ 1; BEGIN { $$max = 0; $$min = 3000; }; \ 1; ++$$n; $$l += ($$F[0] + 1998 - $$F[1])**2; \ 1; $$max = $$F[1] if $$max < $$F[1]; \ 1; $$min = $$F[1] if $$min > $$F[1]; \ 1; } { $$c = $$l / ($$max - $$min)**2; \ 1; printf "average loss = %f\t%.3f\t%.3f\t%.3f\t%.3f\n", $$c/$$n, $$l, $$n, $$min, $$max;') MSDbestmin=1e-2 MSDbestmax=10 MSDnonormbestmin=1e-8 MSDnonormbestmax=1e-5 MSDtimeestimate=15 #--------------------------------------------------------------------- # census-income (KDD) #--------------------------------------------------------------------- census-income.data.gz: @echo "downloading census ..." 1>&2 @wget -q -O - \ http://archive.ics.uci.edu/ml/machine-learning-databases/census-income-mld/$@ \ > $@ census-income.preprocess.data.gz: census-income.data.gz @echo "preprocessing census ..." 1>&2 @zcat $< | \ perl -MScalar::Util -F',' -lane '$$l = pop @F; $$p = $$l =~ /-/ ? -1 : 1; print "$$p $$p|f ", join " ", map { $$isn = Scalar::Util::looks_like_number ($$F[$$_]); $$F[$$_] =~ s/^\s+//; $$F[$$_] =~ s/\s+$$//; $$F[$$_] =~ s/\W/_/g unless $$isn; $$isn ? "@{[$$_+1]}:$$F[$$_]" : "@{[$$_+1]}_$$F[$$_]" } grep { $$F[$$_] =~ /\w/ && ( ! Scalar::Util::looks_like_number ($$F[$$_]) || $$F[$$_] > 0 ) } (0 .. $$#F)' | \ perl -e 'BEGIN { srand 69; }; print map { $$_->[1] } sort { $$a->[0] <=> $$b->[0] } map { [ rand (), $$_ ] } <>;' | gzip \ > $@ census.data: census-income.preprocess.data.gz census.%.nonormlearn: census-income.preprocess.data.gz @${VW} --hash all --loss_function logistic -q ff -l $* \ --adaptive --invariant $< \ -p >(perl -lane ' \ 1; ++$$n; $$l+=1.0 if $$F[0] * $$F[1] < 0; } \ 1; { printf "average loss = %f\t%u\t%u\n", $$l/$$n, $$l, $$n;') census.%.learn: census-income.preprocess.data.gz @${VW} --hash all --loss_function logistic -q ff -l $* $< \ -p >(perl -lane ' \ 1; ++$$n; $$l+=1.0 if $$F[0] * $$F[1] < 0; } \ 1; { printf "average loss = %f\t%u\t%u\n", $$l/$$n, $$l, $$n;') censusbestmin=1e-2 censusbestmax=10 censusnonormbestmin=1e-8 censusnonormbestmax=1e-4 censustimeestimate=5 #--------------------------------------------------------------------- # Statlog (Shuttle) #--------------------------------------------------------------------- shuttle.trn.Z: @echo "downloading Shuttle ..." 1>&2 @wget -q -O - \ http://archive.ics.uci.edu/ml/machine-learning-databases/statlog/shuttle/$@ \ > $@ shuttle.preprocess.data.gz: shuttle.trn.Z @echo "preprocessing Shuttle ..." 1>&2 @zcat shuttle.trn.Z | \ perl -lane '$$l = pop @F; print "$$l |f ", join " ", map { "@{[$$_+1]}:$$F[$$_]" } grep { $$F[$$_] } (0 .. $$#F)' | \ perl -e 'BEGIN { srand 69; }; print map { $$_->[1] } sort { $$a->[0] <=> $$b->[0] } map { [ rand (), $$_ ] } <>;' | gzip \ > $@ shuttle.data: shuttle.preprocess.data.gz shuttle.%.nonormlearn: shuttle.preprocess.data.gz ${VW} --adaptive --invariant -b 22 --hash all -q ff --oaa 7 -l $* -d $< shuttle.%.learn: shuttle.preprocess.data.gz ${VW} -b 22 --hash all -q ff --oaa 7 -l $* -d $< shuttlebestmin=1e-2 shuttlebestmax=10 shuttlenonormbestmin=1e-8 shuttlenonormbestmax=1e-3 shuttletimeestimate=1 #--------------------------------------------------------------------- # CT slices # # normalization doesn't help much #--------------------------------------------------------------------- slice_localization_data.zip: @echo "downloading CTslice ..." 1>&2 @wget -q -O - \ http://archive.ics.uci.edu/ml/machine-learning-databases/00206/$@ \ > $@ slice_localization.preprocess.data.gz: slice_localization_data.zip @echo "preprocessing CTslice ..." 1>&2 @zcat $< | perl -F',' -lane 'BEGIN { scalar <>; }; shift @F; $$l = pop @F; $$l =~ s/\r//; $$l -= 47; print "$$l $$l|f ", join " ", map { ($$F[$$_] eq "-0.25") ? "$${_}_outside" : "$$_:$$F[$$_]" } grep { $$F[$$_] } (0 .. $$#F)' | perl -e 'BEGIN { srand 69; }; print map { $$_->[1] } sort { $$a->[0] <=> $$b->[0] } map { [ rand (), $$_ ] } <>;' | gzip \ > $@ CTslice.data: slice_localization.preprocess.data.gz CTslice.%.nonormlearn: slice_localization.preprocess.data.gz @${VW} --adaptive --invariant -b 22 --hash all -q ff -l $* $< \ -p >(perl -lane ' \ 1; BEGIN { $$max = -3000; $$min = 3000; }; \ 1; ++$$n; $$l += ($$F[0] - $$F[1])**2; \ 1; $$max = $$F[1] if $$max < $$F[1]; \ 1; $$min = $$F[1] if $$min > $$F[1]; \ 1; } { $$c = $$l / ($$max - $$min)**2; \ 1; printf "average loss = %f\t%.3f\t%.3f\t%.3f\t%.3f\n", $$c/$$n, $$l, $$n, $$min, $$max;') CTslice.%.learn: slice_localization.preprocess.data.gz @${VW} -b 22 --hash all -q ff -l $* $< \ -p >(perl -lane ' \ 1; BEGIN { $$max = -3000; $$min = 3000; }; \ 1; ++$$n; $$l += ($$F[0] - $$F[1])**2; \ 1; $$max = $$F[1] if $$max < $$F[1]; \ 1; $$min = $$F[1] if $$min > $$F[1]; \ 1; } { $$c = $$l / ($$max - $$min)**2; \ 1; printf "average loss = %f\t%.3f\t%.3f\t%.3f\t%.3f\n", $$c/$$n, $$l, $$n, $$min, $$max;') CTslicebestmin=1e-2 CTslicebestmax=10 CTslicenonormbestmin=1e-5 CTslicenonormbestmax=1 CTslicetimeestimate=15 #--------------------------------------------------------------------- # common routines #--------------------------------------------------------------------- %.best: %.data @echo "($*) searching for best in-hindsight learning rate for NAG" 1>&2 @printf "WARNING: this step takes about %s minutes\n" $($*timeestimate) @./hypersearch $($*bestmin) $($*bestmax) '$(MAKE)' '$*.%.learn' > $@ %.nonormbest: %.data @echo "($*) searching for best in-hindsight learning rate for AG" 1>&2 @printf "WARNING: this step takes about %s minutes\n" $($*timeestimate) @./hypersearch $($*nonormbestmin) $($*nonormbestmax) '$(MAKE)' '$*.%.nonormlearn' > $@ %.resultsprint: @printf "%20.20s\t%9.3g\t%9.3g\t%9.3g\t%9.3g\n" "$*" $$(cut -f1 $*.best) $$(cut -f2 $*.best) $$(cut -f1 $*.nonormbest) $$(cut -f2 $*.nonormbest) only.%: %.best %.nonormbest all.results.pre %.resultsprint @true .PHONY: all all.results all.results.pre vowpal-wabbit-8.5.0.dfsg1/demo/normalized/README.md000077500000000000000000000060231323713302100216530ustar00rootroot00000000000000normalized online learning demo ------------------------------- These demos show the ability of the normalized learning rule to adapt to varying feature scales, relative to an unnormalized learning rule. For more details on the normalized learning rule see [the paper](http://arxiv.org/abs/1305.6646). ### Instructions ### - `make all.results`: eventually produces a nice table outlining performance of normalized adaptive gradient (NAG) vs. unnormalized adaptive gradient (AG) for a variety of data sets. - **WARNING**: Please be aware that these demos can be network I/O, disk space, and/or CPU intensive. - The complete set of demos can take hours to compute the first time, although the results are cached for subsequent reproduction. - You will see lower progressive loss (regret) for NAG than AG. - You will also see that the optimal learning rate eta* varies less across datasets for NAG than AG. #### Details #### This is organized into individual demos which process a single dataset. The different individual demos are associated with unique make targets. An individual demo will download a data set and learn a predictor under two conditions: using the normalized learning rule aka NAG (this the vw default), and using adaptive gradient without normalization aka AG (invoked via vw arguments `--adaptive --invariant`). For both conditions it will do a hyper-parameter sweep to find the optimal in-hindsight learning rate eta*. Note the context here is online learning, so there is no train/test split; rather what is optimized is progressive loss over the input data. #### About the datasets #### Note when data is pre-normalized or otherwise does not exhibit varying scales, the normalized learning rule has essentially no effect. Therefore the data sets used in this demo have been selected because they exhibit varying scales. The data set [covertype](http://archive.ics.uci.edu/ml/datasets/Covertype) exemplifies how this arises in practice, as it consists of multiple physical measurements with different units. #### Just doing one dataset #### There are individual makefile targets that will just download and compare one dataset. You can compute a subset using a combination of make targets, e.g., `make only.{covertype,shuttle}`. Invocation | Dataset | Time | Disk | Network --- | --- | --- | --- | --- `make only.bank` | [bank](http://archive.ics.uci.edu/ml/datasets/Bank+Marketing) | 2 minutes | 1Mb | 1Mb `make only.census` | [census](http://archive.ics.uci.edu/ml/datasets/Census-Income+%28KDD%29) | 10 minutes | 13Mb | 7Mb `make only.covertype` | [covertype](http://archive.ics.uci.edu/ml/datasets/Covertype) | 12 minutes | 24Mb | 11Mb `make only.CTslice` | [CT Slice](http://archive.ics.uci.edu/ml/datasets/Relative+location+of+CT+slices+on+axial+axis) | 30 minutes | 40Mb | 18Mb `make only.MSD` | [MSD](http://archive.ics.uci.edu/ml/datasets/YearPredictionMSD) | 30 minutes | 500Mb | 256Mb `make only.shuttle` | [Shuttle](http://archive.ics.uci.edu/ml/datasets/Statlog+%28Shuttle%29) | 1 minute | 600Kb | 300Kb vowpal-wabbit-8.5.0.dfsg1/demo/normalized/hypersearch000077500000000000000000000064341323713302100226420ustar00rootroot00000000000000#! /usr/bin/env perl use warnings; use strict; # two parameters: # cmd - a command or reference to an array of command + arguments # timeout - number of seconds to wait (0 = forever) # returns: # cmd exit status (-1 if timed out) # cmd results (STDERR and STDOUT merged into an array ref) sub ExecCmd { my $cmd = shift || return(0, []); my $timeout = shift || 0; # opening a pipe creates a forked process my $pid = open(my $pipe, '-|'); return(-1, "Can't fork: $!") unless defined $pid; if ($pid) { # this code is running in the parent process my @result = (); if ($timeout) { my $failed = 1; eval { # set a signal to die if the timeout is reached local $SIG{ALRM} = sub { die "alarm\n" }; alarm $timeout; @result = <$pipe>; alarm 0; $failed = 0; }; return(-1, ['command timeout', @result]) if $failed; } else { while (<$pipe>) { push @result, $_; print STDERR "."; } } close($pipe); # return exit status, command output return ($? >> 8), \@result; } # this code is running in the forked child process { # skip warnings in this block no warnings; # redirect STDERR to STDOUT open(STDERR, '>&STDOUT'); # exec transfers control of the process # to the command ref($cmd) eq 'ARRAY' ? exec(@$cmd) : exec($cmd); } # this code will not execute unless exec fails! print "Can't exec @$cmd: $!"; exit 1; } sub evaluate ($) { my ($rate) = @_; my $nicerate = sprintf ("%.5g", $rate); print STDERR "trying $nicerate "; my @command = @ARGV; foreach my $c (@command) { $c =~ s/\%/$rate/g; }; my ($status, $rv) = ExecCmd \@command; die "subprocess invocation failed: $!" if $status; my $loss; foreach my $line (@$rv) { next unless $line =~ m%average loss = ([0-9\.]+)%; $loss = $1; } die "failed to parse average loss from vw output: ", join "", @$rv unless defined ($loss); warn " $loss\n"; return $loss; } sub argmin3 ($$$$$$) { my ($a, $fa, $b, $fb, $c, $fc) = @_; if ($fa < $fb) { return $fa < $fc ? ($a, $fa) : ($c, $fc); } else { return $fb < $fc ? ($b, $fb) : ($c, $fc); } } my $lb = shift @ARGV; my $ub = shift @ARGV; my $flb = evaluate ($lb); my $fub = evaluate ($ub); my $tol = 1e-2; my $phi = (1.0 + sqrt (5.0)) / 2.0; my $resphi = 2.0 - $phi; my $mid = $lb + $resphi * ($ub - $lb); my $fmid = evaluate ($mid); while (abs ($ub - $lb) > $tol * abs ($ub + $lb)) { my $x = ($ub - $mid > $mid - $lb) ? $mid + $resphi * ($ub - $mid) : $mid - $resphi * ($mid - $lb); my $fx = evaluate ($x); if ($fx < $fmid) { if ($ub - $mid > $mid - $lb) { $lb = $mid; $mid = $x; $flb = $fmid; $fmid = $fx; } else { $ub = $mid; $mid = $x; $fub = $fmid; $fmid = $fx; } } else { if ($ub - $mid > $mid - $lb) { $ub = $x; $fub = $fx; } else { $lb = $x; $flb = $fx; } } } my ($best, $fbest) = argmin3 ($lb, $flb, $mid, $fmid, $ub, $fub); print "$best\t$fbest\n"; vowpal-wabbit-8.5.0.dfsg1/demo/ocr/000077500000000000000000000000001323713302100170075ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/demo/ocr/Makefile000066400000000000000000000036121323713302100204510ustar00rootroot00000000000000# Handwritten words dataset collected by # Rob Kassel at MIT Spoken Language Systems Group # http://ai.stanford.edu/~btaskar/ocr/ VW = ../../vowpalwabbit/vw # same settings as do-mnist-train VW_OPTS = -b 24 -l 0.1 --nn 40 # http://stackoverflow.com/questions/1541844/joining-elements-of-a-list-in-gnu-make noop= space = $(noop) $(noop) VW_RUN = $(subst $(space),_,$(VW_OPTS)) RM = rm -f # Python > 2.7 is required PYTHON = python help: @echo handwritten words dataset collected by @echo Rob Kassel at MIT Spoken Language Systems Group @echo http://ai.stanford.edu/~btaskar/ocr/ @echo $$ make run letter.data.gz: wget http://ai.stanford.edu/~btaskar/ocr/letter.data.gz letter.names: wget http://ai.stanford.edu/~btaskar/ocr/letter.names letter.vw: ocr2vw.py letter.data.gz letter.names $(PYTHON) $^ $@ $@.test cut -d' ' -f1 $@ | sort | uniq -c | sort -n # category count CATN = 26 letter.model_$(VW_RUN): letter.vw time $(VW) --oaa $(CATN) --final_regressor $@ \ --adaptive --invariant --holdout_off \ --loss_function logistic --passes 14 \ $(VW_OPTS) --data $< -k --cache_file $<.cache_$(VW_RUN) $(RM) $<.cache_$(VW_RUN) letter.predictions_$(VW_RUN): letter.model_$(VW_RUN) time $(VW) --testonly --initial_regressor $< --predictions $@ \ --data letter.vw.test # taken almost verbatim from ../mnist/Makefile CONFUSION='++$$n; $$p=int($$F[0]); $$l=ord($$F[1])-ord("a")+1; \ ++$$c if $$p != $$l; \ ++$$m{"$$l:$$p"}; } { \ print "$* test errors: $$c out of $$n = " . \ sprintf("%.2f%%",100*$$c/$$n) . \ "\nconfusion matrix (rows = truth, columns = prediction):"; \ foreach $$true (1 .. $(CATN)) { \ print join "\t", map { $$m{"$$true:$$_"} || 0 } (1 .. $(CATN)); \ }' %.confusion_$(VW_RUN): %.predictions_$(VW_RUN) @perl -lane $(CONFUSION) $< > $@ @cat $@ run : letter.confusion_$(VW_RUN) clean: $(RM) letter.* .PHONY: clean run vowpal-wabbit-8.5.0.dfsg1/demo/ocr/ocr2vw.py000077500000000000000000000042541323713302100206130ustar00rootroot00000000000000# convert letter.data to letter.vw def read_letter_names (fn): ret = list() with open(fn) as ins: for line in ins: ret.append(line.rstrip()) print "Read %d names from %s" % (len(ret),fn) return ret def find_pixel_start (names): for i in range(len(names)): if names[i].startswith("p_"): return i raise ValueError("No pixel data",names) def data2vw (ifn, train, test, names): lineno = 0 trainN = 0 testN = 0 if ifn.endswith(".gz"): import gzip iopener = gzip.open else: iopener = open id_pos = names.index("id") letter_pos = names.index("letter") pixel_start = find_pixel_start(names) with iopener(ifn) as ins, open(train,"wb") as trainS, open(test,"wb") as testS: for line in ins: lineno += 1 vals = line.rstrip().split('\t') if len(vals) != len(names): raise ValueError("Bad field count", len(vals),len(names),vals,names) char = vals[letter_pos] if len(char) != 1: raise ValueError("Bad letter",char) if lineno % 10 == 0: testN += 1 outs = testS else: trainN += 1 outs = trainS outs.write("%d 1 %s-%s|Pixel" % (ord(char)-ord('a')+1,char,vals[id_pos])) for i in range(pixel_start,len(names)): if vals[i] != '0': outs.write(' %s:%s' % (names[i],vals[i])) outs.write('\n') print "Read %d lines from %s; wrote %d lines into %s and %d lines into %s" % ( lineno,ifn,trainN,train,testN,test) if __name__ == '__main__': import argparse parser = argparse.ArgumentParser(description='Convert letters.data to VW format') parser.add_argument('input',help='path to letter.data[.gz]') parser.add_argument('names',help='path to letter.names') parser.add_argument('train',help='VW train file location (90%)') parser.add_argument('test',help='VW test file location (10%)') args = parser.parse_args() data2vw(args.input,args.train,args.test,read_letter_names(args.names)) vowpal-wabbit-8.5.0.dfsg1/demo/random-noise/000077500000000000000000000000001323713302100206175ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/demo/random-noise/Makefile000066400000000000000000000001301323713302100222510ustar00rootroot00000000000000 all:: @echo === This demo is interactive: hit enter to move to next step ./vw-demo vowpal-wabbit-8.5.0.dfsg1/demo/random-noise/README000066400000000000000000000063511323713302100215040ustar00rootroot00000000000000Demo of vowpal wabbit's ability to separate signal from noise. -------------------------------------------------------------- Prerequisites: -------------- All the components below must be present and located somewhere in your $PATH in order to run the demo. 1) A reasonable Unix-like env, with standard utilities including: GNU less GNU diff bash perl GNU make (not needed if you run 'vw-demo' directly) 2) The following executables (from your vw distribution): vw vw-varinfo 3) The following are used to estimate 'goodness' of model and create density & correlation charts: R ggplot2 (R library. Can be installed from within R see http://ggplot2.org/) Main idea of this demo: ----------------------- As long as noise is random, it cannot bias a model. If it was biased, and could affect the model we're building - it would't be called "noise." The linear expression used in the demo is: y = a + 2b - 5c + 7 (It is a parameter you can actually pass to the main script to change the default). Each demo sequence consists of the following steps: Generate a random train set in which the label y is a perfect linear combination (a + 2b - 5c + 7) of the input features. Generate a model from this random train-set. Generate a _separate_ train-set using the same linear combination formula, but with a different random seed. so the two sets are completely different, even though they are based on the same formula (and model.) Train a model on the train set. Test the model on the test-set, while ignoring the existing labels. The predicted results should equal the actual test set labels. The demo has 3 parts: --------------------- 1) Ideal conditions. No noise added. 2) Global noise in the range [-1, 1] added to the continuous label y in the train-set. 3) Per input-feature noise of +/-50% of each variable range is added to the train set. This affects the label in a 3-modal dispersed way (as shown in a chart produced by the demo) The noise is generated using the standard perl 'rand' function, roughly simulating a uniform distribution, using a simple pseudo-random number generator. We demonstrate that despite the noise, the (almost) perfect model is being learned in each of the 3 parts of the demo. In fact, the small imprecisions in the model are due to us deliberately limiting the precision of the data sets to 6-digits after the decimal point (for brevity and readability purposes). You may change this by passing '-p N' where N is different than 6 to the 'random-poly' script. The whole demo is scripted. --------------------------- All you need to do is: 1) call './vw-demo' (or type 'make') from the shell 2) Hit Enter (repeatedly) to go to the next step In cases the pager (less) is called, you need to hit 'q' to exit the pager. Read the presentation: ---------------------- A presentation (set of slides) including this demo in both pdf and ppt formats can be found here: http://finance.yendor.com/ML/VW/ -- ariel faigon - aug 2013 vowpal-wabbit-8.5.0.dfsg1/demo/random-noise/distrib.r000077500000000000000000000046001323713302100224450ustar00rootroot00000000000000#!/usr/bin/Rscript # --vanilla # # distrib.r: # utility to plot distribution/density of a numeric data-set column # # Usage: # distrib.r data_file ["optional chart title string"] # where data_file contains the numeric vector, a number per line. # # -- where to look for R libraries # .libPaths(c('~/local/lib/R', # '/usr/lib/R/library', # '/usr/lib/R/site-library' # )) suppressPackageStartupMessages(library(ggplot2)) ratio = 1.61803398875 W = 4 H = W / ratio DPI = 200 FONTSIZE = 9 MyGray = 'grey50' title.theme = element_text(family="FreeSans", face="bold.italic", size=FONTSIZE-2) x.title.theme = element_text(family="FreeSans", face="bold.italic", size=FONTSIZE-2, vjust=-0.1) y.title.theme = element_text(family="FreeSans", face="bold.italic", size=FONTSIZE-2, angle=90, vjust=0.2) x.axis.theme = element_text(family="FreeSans", face="bold", size=FONTSIZE-2, colour=MyGray) y.axis.theme = element_text(family="FreeSans", face="bold", size=FONTSIZE-2, colour=MyGray) legend.theme = element_text(family="FreeSans", face="bold.italic", size=FONTSIZE-1, colour="black") eprintf <- function(...) cat(sprintf(...), sep='', file=stderr()) argv <- commandArgs(trailingOnly = TRUE) csvfile <- argv[1] title <- ifelse(! is.na(argv[2]), argv[2], 'vw demo: random expression distribution') Ys <- read.csv(csvfile, header=F, col.names='Ys') d <- data.frame(Ys=Ys) Y_labels <- function(yrange) { the.min <- as.integer(floor(yrange[1])) the.max <- as.integer(ceiling(yrange[2] + 1)) seq(from=the.min, to=the.max, by=1) } # geom_histogram(binwidth=.5, alpha=.5, position="identity") # geom_histogram(fill='#3377ff', # binwidth=.01, alpha=.4, stat='density') + g <- ggplot(data=d, aes(x=Ys)) + geom_density(fill='#3377ff', alpha=0.4, lwd=0.2) + scale_x_continuous(breaks=Y_labels(range(Ys))) + ggtitle(title) + xlab(NULL) + theme( plot.title=title.theme, axis.title.y=y.title.theme, axis.title.x=x.title.theme, axis.text.x=x.axis.theme, axis.text.y=y.axis.theme ) pngfile <- sprintf("%s.density.png", csvfile) ggsave(g, file=pngfile, width=W, height=H, dpi=DPI) vowpal-wabbit-8.5.0.dfsg1/demo/random-noise/random-poly000077500000000000000000000140711323713302100230110ustar00rootroot00000000000000#!/usr/bin/perl -w # vim: ts=4 sw=4 expandtab # # Generate a random data-set matching a linear expression # TODO: support real polynomial expressions with x^n # use Getopt::Std; use Scalar::Util qw(looks_like_number); use vars qw($opt_v $opt_D $opt_n $opt_c $opt_t $opt_s $opt_p $opt_w $opt_r $opt_R); my $DefaultN = 10; # number of generated examples my @DeafultExpr = ('a'); my $DefaultFloatPrec = 6; # of decimal points for floating point values my $Sep = ' '; my $ConstStr = '_CONST_'; sub v { return unless $opt_v; if (@_ == 1) { print STDERR @_; } else { printf STDERR @_; } } # Undocumented, for debugging sub D { return unless $opt_D; if (@_ == 1) { print STDERR @_; } else { printf STDERR @_; } } sub usage(@) { print STDERR @_, "\n" if @_; die "Usage: $0 [Options] expression... Options: -v verbose -n Generate rows (data-set examples) -s Call srand() at start -p

Set data-set feature precision to

digits -w Add weight to examples -r Add uniform random noise in range [, ] -R Add per-feature uniform random noise in range [, ] -c output in CSV format (label is 1st column) -t output in TSV format (label is 1st column) Default format is VW (vowpal-wabbit) Examples of expression: 2x + 5y - 4 w +x +y +z ... "; } # # Convert to a list of easily parsable terms, e.g: # "2x+70y-1" => 2x +70y -1 # sub argv_2_terms(@) { my $one_arg = "@_"; $one_arg =~ tr/ //d; $one_arg =~ s/([-+])/ $1/g; my @term_list = split(' ', $one_arg); v("argv_2_terms($one_arg) -> @term_list\n"); @term_list; } sub init { $0 =~ s{.*/}{}; getopts('vDn:ctwp:s:r:R:'); $Sep = ($opt_c) ? ',' : $opt_t ? "\t" : ' '; $opt_n = $DefaultN unless ($opt_n); $opt_p = $DefaultFloatPrec unless (defined $opt_p); # srand($opt_s ? $$^time : 0); srand($opt_s ? $opt_s : 0); if ($opt_r) { my ($min, $max) = split(/[\s,;]+/, $opt_r); usage("-r: bad expresssion, expecting numeric ,") unless (looks_like_number($min) && looks_like_number($max)); usage("-r: bad expresssion, > ") unless ($min <= $max); # Turn opt_r into a function using currying $opt_r = sub { my $r = shift; return $min + ($r * ($max - $min)); } } if ($opt_R) { my ($min, $max) = split(/[\s,;]+/, $opt_R); usage("-R: bad expresssion, expecting numeric ,") unless (looks_like_number($min) && looks_like_number($max)); usage("-R: bad expresssion, > ") unless ($min <= $max); my $range = ($max - $min); # Turn opt_R into a function using currying $opt_R = sub { my $r = shift; my $r2 = rand(1.0); # make the random noise # - Make it a multiplier on the original $r # - multiplier is a random $r2 point in the range (max - min) # - Make it work both ways my $noise = $min + $r2 * $range; my $retval = $r * $noise; v("\topt_R: r2=%g*(min,max)=(%g,%g) noise=%g * \$r=%g => %g\n", $r2, $min, $max, $noise, $r, $retval); return $retval; } } @ARGV = @DeafultExpr unless (@ARGV); @ARGV = argv_2_terms(@ARGV); } sub parse_expr(@) { my @expr = (); foreach my $term (@_) { if ($term =~ /^([-+]?)(\d*(?:\.\d*)?)\*?([A-Za-z]*)$/) { my ($sign, $coeff, $varname) = ($1, $2, $3); $sign = ($sign eq '-') ? -1.0 : 1.0; $coeff = 1.0 unless ($coeff); $coeff *= $sign; $varname = $ConstStr unless ($varname); push(@expr, [$coeff, $varname]); v("parse_expr: [coeff, varname]: [%s, %s]\n", $coeff, $varname); } else { usage("bad term: '$term': expecting : e.g. 2.3x"); } } @expr; } sub gen_dataset(@) { my (@expr) = @_; for ($i = 1; $i <= $opt_n; $i++) { my $result = 0; my $feature_str = ''; for ($nvar = 0; $nvar < @expr; $nvar++) { my ($coeff, $varname) = @{$expr[$nvar]}; my ($r, $term_value); if ($varname eq $ConstStr) { $r = $coeff; $term_value = $coeff; } else { $r = rand(1.0); $term_value = $r; if (defined $opt_R) { my $random_add = $opt_R->($r); v("\topt_R: r: %g -> random_add: %g\n", $r, $random_add); $term_value += $random_add; } $term_value *= $coeff; } v("\tgen_dataset: [%s%s] r:%.*f => %.*f\n", $coeff, $varname, $opt_p, $r, $opt_p, $term_value); if ($varname ne $ConstStr) { # Don't add constant to the input features # Add it only to the label $feature_str .= ($opt_c || $opt_t) ? sprintf("%s%.*f", $Sep, $opt_p, $r) : sprintf("%s%s:%.*f", $Sep, $varname, $opt_p, $r); } $result += $term_value; } if (defined $opt_r) { my $random_noise = $opt_r->(rand(1.0)); # v("random_noise: %g\n", $random_noise); $result += $random_noise; } if ($opt_c || $opt_t) { printf "%.*f%s\n", $opt_p, $result, $feature_str; } elsif ($opt_w) { # -- add weight printf "%.*f %s %s|f%s\n", $opt_p, $result, 1, $i, $feature_str; } else { printf "%.*f '%s|f%s\n", $opt_p, $result, $i, $feature_str; } } } # -- main init(); my @expr = parse_expr(@ARGV); gen_dataset(@expr); vowpal-wabbit-8.5.0.dfsg1/demo/random-noise/vw-demo000077500000000000000000000302601323713302100221240ustar00rootroot00000000000000#!/bin/bash # # Vowpal Wabbit interactive demo of noise-resistance # # Requires the following to be installed on your machine: # # 1) vw - the vowpal-wabbit executable # 2) R + ggplot2 - for all the beautiful charts # 3) A few scripts included with this one: # 3a) random-poly - a perl script for generating random data-sets # 3b) distrib.r - Density distribution plot utility, written in R # 3c) x-vs-y.r - X vs Y correleation plot utility, written in R # export PATH=.:$PATH Pager='less' ImgViewCandidates="gwenview display irfanview xee preview" ImgViewer= Poly='a + 2b - 5c + 7' find_ggplot2() { err=`Rscript -e 'library(ggplot2)' 2>&1 | grep 'Error.*ggplot'` case "$err" in *rror*) echo "Couldn't find the R ggplot2 library. Is it installed?" 1>&2 echo -- "$err" 1>&2 exit 1 ;; esac } find_image_viewer() { # Please add your favorite OS image viewer here for exe in $ImgViewCandidates; do case `which $exe` in '') : keep going... ;; *) ImgViewer=$exe : found image viewer: $ImgViewer break ;; esac done case "$ImgViewer" in '') echo "Sorry: coudn't find an image viewer in PATH=$PATH Please add your viewer to 'ImgViewCandidates' in the '$0' script." 1>&2 exit 1 ;; esac } check_prereqs() { missing=0 find_image_viewer for exe in vw R Rscript random-poly distrib.r x-vs-y.r; do case `which $exe` in '') echo "$0: can't find $exe in PATH=$PATH - please install it" missing=$(($missing+1)) ;; *) : found $exe - cool ;; esac done case $missing in 0) : ;; *) exit 1 ;; esac find_ggplot2 } # # demo_cmd is the work-horse of our presentation. # 'main' can be simply a sequence of multiple calls to it. # It has 3 goals: # 1) Ensure we get all the little details right and never # make a mistake during the actual presentation # 2) Save time typing stuff # 3) Anyone else can reproduce what we did perfectly in their # own env. # # demo_cmd [options] 'header/explanation string' 'command string' # options: # -p don't pause for user to hit [enter] # -h don't print the header-string # -s don't advance the step # -e don't echo the command (be silent), just execute # -c don't execute the command # demo_cmd() { # by default we do all of them opt_p=1; opt_h=1; opt_s=1; opt_e=1 opt_c=1 # Must initialize OPTIND since it doesn't reset between # calls to 'demo_cmd()'! OPTIND=1 while getopts 'phsec' opt; do case "$opt" in p) opt_p= ;; h) opt_h= ;; s) opt_s= ;; e) opt_e= ;; c) opt_c= ;; esac done shift $((OPTIND-1)) header="$1" cmd="$2" # echo "demo_cmd: args: |$@| header=|$header| cmd=|$cmd| OPTIND=$OPTIND" case $opt_s in 1) step=$(($step+1)) ;; esac case $opt_h in 1) echo "=== $step: $header" ;; esac case $opt_p in 1) # read the command-line in, but allow real-time edits # via GNU readline read -ep "\$ " -i "$cmd" ans ;; *) case $opt_e in 1) # If we have no readline/prompt we need to print # the command so it can be seen by the audience echo -n "\$ $cmd" ;; esac ;; esac case $opt_c in 1) case $opt_p in 1) eval "$ans" ;; *) eval "$cmd" ;; esac echo ;; esac } label_column() { data_file="$1" label_file="$2" cut -d' ' -f1 $data_file > $label_file } y_density() { data_file=$1 chart_title=$2 label_file="Ys/$data_file" label_column "$data_file" "$label_file" distrib.r $label_file "$chart_title" $ImgViewer $label_file.density.png 2>/dev/null } clean_slate() { /bin/rm -f r.* Ys 2>/dev/null mkdir -p Ys } # # demo_session # full session of random-data-generation, training, testing... # demo_session() { mode="$1" step=0 clean_slate # --- train-set generation case $mode in globalnoise) rand='-r -1,1'; msg=' (w/ global noise)' ;; varnoise) rand='-R -.5,.5'; msg=' (w/ per-var noise)' ;; regularize) rand='-R -.5,.5'; msg=' (w/ per-var noise)' ;; clean) rand='-r 0,0'; msg='' ;; # no noise added esac demo_cmd "Generate a training data-set$msg & inspect it (-n N is number of data-points (examples) -pN is data precision -r min,max is global added noise -R min,max is per-variable added noise):" \ "random-poly -n 50000 -p6 $rand $Poly > r.train" case $mode in 'clean') # Only do this the 1st time, othewise it is getting tedious demo_cmd -s "inspect the training-set (use 'q' to exit $Pager):" \ "$Pager r.train" ;; esac demo_cmd "Visualize train-set Ys (labels) density [~5 secs to generate chart]:" \ "y_density r.train 'Train-set random expression distribution: $Poly'" case $mode in # --- in the case where we added noise, # --- add a step of showing how big is the noise *noise) echo '+-----------------------------------------------------+' echo '| visualize the added random noise |' echo '+-----------------------------------------------------+' # -- Prepare the reference Ys (without the noise) case $mode in globalnoise) random-poly -n 50000 -p6 -r 0,0 $Poly | \ cut -d' ' -f1 > Ys/r.train.nonoise ;; varnoise) random-poly -n 50000 -p6 -R 0,0 $Poly | \ cut -d' ' -f1 > Ys/r.train.nonoise ;; esac demo_cmd -p "Generate plot of clean vs NOISY Ys (labels)" \ 'x-vs-y.r Ys/r.train.nonoise Ys/r.train X-vs-Y.png' demo_cmd -p "View plot of CLEAN (X) vs NOISE-filled (Y) values " \ "$ImgViewer X-vs-Y.png 2>/dev/null" ;; esac # --- Training case $mode in globalnoise) vw_args='' msg1='Train: let VW build a model on noisy -1/+1 train-set' msg2='Train: look at the model weights' ;; varnoise) vw_args='' msg1='Train: let VW build a model (w/ per var noise)' msg2='Train: look at the model weights (w/ per var noise)' ;; regularize) vw_args='--l2 0.000001' msg1='Train: let VW build a model (w/ anti-noise --l2)' msg2='Train: look at the model weights (w/ anti-noise --l2)' ;; clean) # vanilla vw_args='-l 5' msg1='Train: let VW build a model from the train-set' msg2='Train: look at the model weights' ;; esac demo_cmd "$msg1" "vw -k $vw_args r.train -f r.model" echo '+------------------------------------------------------------------+' echo '# Notice how fast training took to complete (about 0.1 sec).' echo '# vw is faster processing data than all other programs in this demo.' echo '#' echo '# Since learning is faster than IO, and runs in a separate thread,' echo '# vw training speed is limited only by the time to read the data.' echo '+------------------------------------------------------------------+' demo_cmd "$msg2" "vw-varinfo -k $vw_args r.train" echo '+------------------------------------------------------------------+' echo '# Notice how accurate the model is: model weights are exactly,' echo "# or very close to our target linear expression: $Poly" echo '+------------------------------------------------------------------+' # --- test-set generation demo_cmd "Generate a test data-set (note different random seed: -s)" \ "random-poly -n 50000 -p6 -s 1313131 $Poly > r.test" demo_cmd "Show that train and test data-sets are different" \ 'diff <(head -9 r.train) <(head -9 r.test)' demo_cmd "Visualize test-set Ys (labels) density [~5 sec to generate chart]:" \ "y_density r.test 'Test-set random expression distribution: $Poly'" demo_cmd -p "Clear the Ys (labels) from the test-set" \ 'perl -i -pe "s/\S+/0/" r.test' case $mode in 'clean') # Only do this the 1st time, othewise it is getting tedious demo_cmd -s "inspect test-set to see Ys (labels) are gone (hit 'q' to exit $Pager):" \ "$Pager r.test" esac # --- prediction of test-set Ys using trained-model demo_cmd "Predict: VW uses the model to predict the test-set Ys (labels)" \ 'vw -t -i r.model r.test -p r.predict' echo '+-----------------------------------------------------------+' echo '# Since Ys (labels) have been zeroed - the reported error' echo '# is large even though predictions are, in fact, accurate.' echo '# We are also running vw with "-t" (test-only) so no weights' echo '# are being updated in-memory during the prediction run.' echo '+-----------------------------------------------------------+' demo_cmd -p "Extract 1st column (Ys) of prediction set (label_column is an internal func defined in $0)" \ 'label_column r.predict Ys/r.predict' # --- Check prediction (vs. actual) quality # textual eyeball inspection demo_cmd "Compare predictions with actual values side-by-side Note how close they are, since the model weights are near-perfect:" \ "diff -y -W 24 Ys/r.predict Ys/r.test | $Pager" demo_cmd -p "Plot predictions vs actual (test) values [~5 secs to generate chart]:" \ 'x-vs-y.r Ys/r.predict Ys/r.test X-vs-Y.png' demo_cmd "Look at plot of predicted vs actual (test) values:" \ "$ImgViewer X-vs-Y.png 2>/dev/null" } # # -- main # check_prereqs case "$@" in # support passing an initial expression for the whole demo # from the command line *[0-9a-zA-Z]*) Poly="$@" ;; esac echo '+-----------------------------------------------------------------+' echo '| Demo of vw ability to separate signal from noise |' echo '| |' echo '| 1) Create a random data-set & learn from it (perfectly). |' echo '| 2) Add global noise to each example, and finally, |' echo '| 3) Add a separate noise component to each input feature. |' echo '| |' echo '| Goal: demonstrate how vw creates near perfect models |' echo '| despite various forms of noise. |' echo '| At each of the 3 steps we visualized he data-set label density, |' echo '| the noise, and the model prediction quality using R+ggplot2. |' echo '+-----------------------------------------------------------------+' echo '+-----------------------------------------------------------------+' echo '| 1) First session warm-up: in a "perfect" world (no noise)... |' echo '+-----------------------------------------------------------------+' demo_session clean echo '+-----------------------------------------------------------------+' echo '| 2) Repeat session + added GLOBAL random noise |' echo '+-----------------------------------------------------------------+' demo_session globalnoise echo '+-----------------------------------------------------------------+' echo '| 3) Repeat session + added PER VARIABLE random noise |' echo '+-----------------------------------------------------------------+' demo_session varnoise echo " -----> Q.E.D" # --- Demo using regularization # Not done here. We need a more challenging data-set to # demonstrate effective use of regularization. # echo '+-----------------------------------------------------------------+' # echo '| Repeat session + added anti-random noise (w/ --l2) |' # echo '+-----------------------------------------------------------------+' # demo_session regularize vowpal-wabbit-8.5.0.dfsg1/demo/random-noise/x-vs-y.r000077500000000000000000000054741323713302100221620ustar00rootroot00000000000000#!/usr/bin/Rscript # --vanilla # # Utility to plot two data-set numeric columns against each other, # and generate a X-vs-Y chart + pearson correlation between the two. # # Usage: x-vs-y.r data_file1 data_file2 pngfile # Where: # data_file1 & data_file2 # each contains one data column from some data-set. # Columns can be easily pre-extracted into the files using # cut -d... -f... dataset # # pngfile # is the output chart # # -- where to look for R libraries # .libPaths(c('~/local/lib/R', # '/usr/lib/R/library', # '/usr/lib/R/site-library' # )) suppressPackageStartupMessages(library(ggplot2)) one_column <- function(filename, sep, fieldno) { cmd <- sprintf("cut -d '%s' -f %d '%s'", sep, fieldno, filename) read.table(pipe(cmd), header=F)[[1]] } ratio = 1 W = 6 H = W / ratio DPI = 200 FONTSIZE = 12 MyGray = 'grey50' title.theme = element_text(family="FreeSans", face="bold.italic", size=FONTSIZE) x.title.theme = element_text(family="FreeSans", face="bold.italic", size=FONTSIZE, vjust=-0.1) y.title.theme = element_text(family="FreeSans", face="bold.italic", size=FONTSIZE, angle=90, vjust=0.2) x.axis.theme = element_text(family="FreeSans", face="bold", size=FONTSIZE-2, colour=MyGray) y.axis.theme = element_text(family="FreeSans", face="bold", size=FONTSIZE-2, colour=MyGray) legend.theme = element_text(family="FreeSans", face="bold.italic", size=FONTSIZE-1, colour="black") eprintf <- function(...) cat(sprintf(...), sep='', file=stderr()) argv <- commandArgs(trailingOnly = TRUE) # Xs <- read.csv(argv[1], header=F, col.names='X', colClasses=c('numeric')) Xs <- as.numeric(one_column(argv[1], sep=' ', 1)) # Ys <- read.csv(argv[2], header=F, col.names='Y', colClasses=c('numeric')) Ys <- as.numeric(one_column(argv[2], sep=' ', 1)) pearson <- as.numeric(cor(Xs, Ys)) eprintf("\nPearson Correlation: %.12f\n", pearson) d <- data.frame(X=Xs, Y=Ys) title <- sprintf('vw demo: expected vs actual values\nPearson correlation: %.12f', pearson) adaptive.alpha <- 0.02 + (abs(2.0 - (2.0 * pearson))) * 0.3 # eprintf("adaptive.alpha=%g\n", adaptive.alpha) g <- ggplot(data=d, aes(x=X, y=Y), ) + geom_point(shape=20, alpha=adaptive.alpha, size=0.4) + ggtitle(title) + theme( plot.title=title.theme, axis.title.y=y.title.theme, axis.title.x=x.title.theme, axis.text.x=x.axis.theme, axis.text.y=y.axis.theme ) pngfile <- ifelse(exists(argv[3]) && nchar(argv[3]) > 0, argv[3], 'X-vs-Y.png') # eprintf("ggsave: pngfile=%s\n", pngfile) ggsave(g, file=pngfile, width=W, height=H, dpi=DPI) vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/000077500000000000000000000000001323713302100205055ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/Makefile000066400000000000000000000016651323713302100221550ustar00rootroot00000000000000.SECONDARY: bits?=26 passes?=80 defaults: defaults.vw aloi_test.vw.gz zcat $(word 2,$^) | sed 's/|/|f/' | ../../vowpalwabbit/vw -t -i $< defaults.vw: aloi_train.vw.gz zcat $< | sed 's/|/|f/' | ../../vowpalwabbit/vw -f $@ -k -c --passes $(passes) --recall_tree 1000 -b $(bits) --loss_function logistic -q'\x88': -l 1 tuned: tuned.vw aloi_test.vw.gz zcat $(word 2,$^) | sed 's/|/|f/' | ../../vowpalwabbit/vw -t -i $< tuned.vw: aloi_train.vw.gz zcat $< | sed 's/|/|f/' | ../../vowpalwabbit/vw -f $@ -k -c --passes $(passes) --recall_tree 1000 -b $(bits) --loss_function logistic -q'\x88': -l 0.85 --randomized_routing 1 --max_depth 12 --max_candidates 58 --bern_hyper 0.01 aloi_test.vw.gz: wget http://lowrank.net/nikos/aloi_test.vw.gz aloi_train.vw.gz: wget http://lowrank.net/nikos/aloi_train.vw.gz oaa: oaa.vw aloi_test.vw.gz vw -t -i $^ oaa.vw: aloi_train.vw.gz vw -f $@ -k -c --passes $(passes) --oaa 1000 $< -b $(bits) -l 1.86 vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/README.md000066400000000000000000000055631323713302100217750ustar00rootroot00000000000000Recall tree demo ------------------------------- This demo exercises the recall tree reduction for logarithmic time multiclass classification. The dataset used is [ALOI](http://aloi.science.uva.nl/), which only has 1000 classes in it. Due to constant factor overhead, I wouldn't recommend using recall tree unless you have at least 10,000 classes; but it's good to have a demo that runs quickly without a lot of memory. The demo targets are: - `make defaults`: trains and tests using what are essentially the defaults for recall tree (see below). Eventually yields 11.2% test error. - `make tuned`: trains and tests using hyperparameters found via random probe hyperparameter tuning. Eventually yields 9.9% test error. - `make oaa`: trains and tests one-against-all using hyperparameters found via random probe hyperparameter tuning. Yields 12.1% test error (in less time than recall tree, it turns out). ### Recall Tree for your problem ### - As indicated, if you have less than 10000 classes, you should probably be using OAA. To accelerate training you can use negative gradient subsampling and multicore training via hogwild. - If you're still reading, you have more than 10000 classes. Therefore you will need many bits in your predictor. Use as many as you can afford computationally. - If your features are all binary then try disabling normalized updates via `--adaptive --invariant`, which is like having an extra bit in your predictor without the memory cost. - Logistic loss (`--loss_function logistic`) always works better, as far as I can tell. - There are extra features, consisting of the identities of the nodes in the routing tree, which are added to the example before passing to the underlying binary classifier. - These are located in namespace '\x88'. So, for example, you can interact them with other namespaces as in this demo (see Makefile: this is what `-q '\x88':` is doing.) - For problems with lots of features, interacting with all the path features can be both a computational and statistical drag. Enabling the option `--node_only` only generates a single feature corresponding to the identity of the leaf node in the routing tree, which can be better under these conditions. - Typically increasing the number of candidates per leaf `--max_candidates` from the default will improve accuracy at the cost of additional computation. - Computational overhead increases with deeper trees, but (test) accuracy does not necessarily increase. Therefore you have to play around with `--max_depth` and `--bern_hyper`. - Increasing `--bern_hyper` discourages deeper trees, decreasing it encourages deeper trees. - `--max_depth` is a hard limit. - Randomized routing `--randomized_routing` is a regularizer we came up with which sometimes works well. It's worth trying, but note that it will make your loss on the training set appear worse, so you'll need to assess on test data. vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/imagenet/000077500000000000000000000000001323713302100222765ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/imagenet/Makefile000066400000000000000000000077421323713302100237500ustar00rootroot00000000000000SHELL=/bin/zsh numlabels=21841 bits?=30 .SECONDARY: training.txt.gz: wget --tries=100 -c http://hunch.net/~jl/datasets/imagenet/training.txt.gz # ln -s /data/jcl/training.txt.gz testing.txt.gz: wget --tries=100 -c http://hunch.net/~jl/datasets/imagenet/testing.txt.gz # ln -s /data/jcl/testing.txt.gz full_oaa_hyper%: full_oaa_hyper%.vw testing.txt.gz time ./do-oaa-hogwild $(word 2,$^) -i $< -t full_oaa_hyper%.vw: training.txt.gz perl -le 'BEGIN { srand $*; }; \ 1; $$eta = 0.1 + 3.9 * rand (); \ 1; print join " ", $$eta;' | \ while read eta ; \ do ( \ echo $$eta ; \ ./do-oaa-hogwild $< -f $@ --oaa $(numlabels) --oaa_subsample 1000 -b $(bits) --loss_function logistic -l $$eta ; \ ) \ done #> full_oaa_hyper$*.train 2>&1; \ oaa_hyper%: training.txt.gz perl -le 'BEGIN { srand $*; }; \ 1; $$eta = 0.1 + 3.9 * rand (); \ 1; print join " ", $$eta;' | \ while read eta ; \ do ( \ echo $$eta ; \ vw <(zcat $< | head -500000) --oaa $(numlabels) --oaa_subsample 1000 -b $(bits) --loss_function logistic -l $$eta ; \ ) > $@ 2>&1; \ done timefufull_hyper%: full_hyper%.vw testing.txt.gz (time vw -t -i $< <(zcat $(word 2,$^) | head -10000); time vw -t -i $< <(zcat $(word 2,$^) | head -20000)) > $@ 2>&1 full_hyper%: full_hyper%.vw testing.txt.gz (time vw -t -i $(word 1,$^) <(zcat $(word 2,$^) | sed 's/|/|b/')) > $@ 2>&1 full_hyper%.vw: training.txt.gz perl -le 'BEGIN { srand $*; }; \ 1; $$eta = 0.1 + 3.9 * rand (); \ 1; $$depth = int (11 + 6 * rand ()); \ 1; $$mc = int (20 + 80 * rand ()); \ 1; $$log_hyper = log (0.01) + (log (10) - log (0.01)) * rand (); \ 1; $$rr = rand () > 0.5 ? 1 : 0; \ 1; $$dq = rand () > 0.5 ? "a" : "b"; \ 1; print join " ", $$eta, $$depth, $$mc, exp ($$log_hyper), $$rr, $$dq;' | \ while read eta depth mc hyper rr dq; \ do ( \ echo $$eta $$depth $$mc $$hyper $$rr $$dq; \ vw -f $@ --recall_tree $(numlabels) <(zcat $< | sed 's/|/|b/') -b $(bits) --loss_function logistic -l $$eta --max_depth $$depth --max_candidates $$mc --bern_hyper $$hyper --link glf1 --randomized_routing $$rr --node_only 1 -q '\x88'$$dq; \ ) > full_hyper$*.train 2>&1; \ done hyper%: training.txt.gz perl -le 'BEGIN { srand $*; }; \ 1; $$eta = 0.1 + 3.9 * rand (); \ 1; $$depth = int (11 + 6 * rand ()); \ 1; $$mc = int (20 + 80 * rand ()); \ 1; $$log_hyper = log (0.01) + (log (10) - log (0.01)) * rand (); \ 1; $$rr = rand () > 0.5 ? 1 : 0; \ 1; $$dq = rand () > 0.5 ? "a" : "b"; \ 1; print join " ", $$eta, $$depth, $$mc, exp ($$log_hyper), $$rr, $$dq;' | \ while read eta depth mc hyper rr dq; \ do ( \ echo $$eta $$depth $$mc $$hyper $$rr $$dq; \ vw --recall_tree $(numlabels) <(zcat $< | sed 's/|/|b/' | head -500000) -b $(bits) --loss_function logistic -l $$eta --max_depth $$depth --max_candidates $$mc --bern_hyper $$hyper --link glf1 --randomized_routing $$rr --node_only 1 -q '\x88'$$dq; \ ) > $@ 2>&1; \ done log_multi: log_multi.vw testing.txt.gz time vw -t -i $^ log_multi.vw: training.txt.gz time vw -f $@ --log_multi $(numlabels) $< -b $(bits) --loss_function logistic -l 1 recall_tree: recall_tree.vw testing.txt.gz time vw -t -i $(word 1,$^) <(zcat $(word 2,$^) | sed 's/|/|b/') recall_tree.vw: training.txt.gz time vw -f $@ --recall_tree $(numlabels) <(zcat $< | sed 's/|/|b/') -b $(bits) --loss_function logistic -l 1 --node_only 1 -q '\x88'b #--link glf1 --randomized_routing 1 #$(randomized_routing) oaahogwild.vw: training.txt.gz time ./do-oaa-hogwild $< -f $@ --oaa $(numlabels) --oaa_subsample 1000 -b $(bits) --loss_function logistic -l 1 oaahogwild: oaahogwild.vw testing.txt.gz (time vw -t -i $^) > $@ 2>&1 #(time ./do-oaa-hogwild -t -i $^) > $@ 2>&1 timefuoaahogwild: oaahogwild.vw testing.txt.gz (time vw -t -i $< <(zcat $(word 2,$^) | head -10000); time vw -t -i $< <(zcat $(word 2,$^) | head -20000)) > $@ 2>&1 vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/imagenet/README.md000066400000000000000000000003771323713302100235640ustar00rootroot00000000000000Recall tree demo for imagenet ------------------------------- This demo exercises the recall tree reduction for logarithmic time multiclass classification on the imagenet dataset. It takes a long time to run, a lot of memory, and a lot of disk space. vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/imagenet/do-oaa-hogwild000077500000000000000000000040361323713302100250220ustar00rootroot00000000000000#! /bin/zsh nukeem() { \ trap - INT QUIT TERM pkill -9 -f 'vw.*--port 26543' } if which netcat >/dev/null 2>/dev/null; then netcat=netcat elif which nc >/dev/null 2>/dev/null; then netcat=nc else echo "Install netcat or nc" >&2 exit 1 fi data="$1" shift { vw "$@" --num_children 24 --port 26543 2>&1 | perl -lane 'print $_ unless $F[2] =~ /^\d$/ && $c{$F[2]}++;' } & trap 'nukeem; exit 1' INT QUIT TERM while ! $netcat -z localhost 26543 do sleep 1 done sleep 5 zcat $data | \ ./map \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) \ >($netcat localhost 26543 > /dev/null) pkill -f 'vw.*--port 26543' while test "x$1" = "x-f" && test ! -s "$2" do sleep 1 done vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/imagenet/map000077500000000000000000000010571323713302100230040ustar00rootroot00000000000000#! /usr/bin/env perl use warnings; use strict; use IO::File; use IO::Select; my @mappers = map { $_->autoflush (1); $_ } map { new IO::File $_, "w" or die "$_: $!" } @ARGV; my $bufsize = 1; my @buf = grep { defined } map { scalar } (1 .. $bufsize); my $n = 0; while (@buf) { my $fh = $mappers[$n % @mappers]; my @ready = IO::Select->new ($fh)->can_write (20000); if (@ready) { print $fh @buf; @buf = grep { defined } map { scalar } (1 .. $bufsize); } ++$n; } vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/ltcb/000077500000000000000000000000001323713302100214315ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/ltcb/Makefile000066400000000000000000000252641323713302100231020ustar00rootroot00000000000000SHELL=/bin/zsh .SECONDARY: numlabels=80000 all: enwik8.train.vw.gz sweep_oaa%: sweep_oaa%.vw enwik8.test.vw.gz (time vw -t -i $(word 1,$^) <(zcat $(word 2,$^) | perl -ne '($$l,$$b)=split /\s/, $$_, 2; $$l = $(numlabels)+2-$$l; $$l = 1+$* if $$l > $*; print "$$l $$b"')) > $@ 2>&1 sweep_oaa%.vw: enwik8.train.vw.gz (time vw -f $@ --oaa $$((1+$*)) --oaa_subsample 1000 <(zcat $< | perl -ne '($$l,$$b)=split /\s/, $$_, 2; $$l = $(numlabels)+2-$$l; print "$$l $$b" if $$l <= $*') -q bc --cubic bcd -b 30 --loss_function logistic -l 1) > sweep_oaa$*.train 2>&1 full_inorder_log_multihyper%.vw: bits=32 full_inorder_log_multihyper%.vw: enwik9.inorder.train.vw.gz perl -le 'BEGIN { srand $*; }; \ 1; $$eta = 0.1 + 3.9 * rand (); \ 1; $$nodes = int (10000 + 40000 * rand ()); \ 1; $$loss = rand () > 0.5 ? "squared" : "logistic"; \ 1; $$swap = 2 ** (int (4 * rand ())); \ 1; $$quad = rand () > 0.5 ? "z" : "b"; \ 1; $$cubic = rand () > 0.5 ? "z" : "b"; \ 1; print join " ", $$eta, $$nodes, $$loss, $$swap, $$quad, $$cubic;' | \ while read eta nodes loss swap quad cubic; \ do ( \ echo $$eta $$nodes $$loss $$swap $$quad $$cubic; \ time vw -f $@ --log_multi $$(( $(numlabels)+1 )) $< -q c$$quad --cubic cd$$cubic -b $(bits) --loss_function $$loss -l $$eta --swap_resistance $$swap; \ ) > full_inorder_log_multihyper$*.train 2>&1; \ done full_log_multihyper%.vw: bits=32 full_log_multihyper%.vw: enwik9.train.vw.gz perl -le 'BEGIN { srand $*; }; \ 1; $$eta = 0.1 + 3.9 * rand (); \ 1; $$nodes = int (10000 + 40000 * rand ()); \ 1; $$loss = rand () > 0.5 ? "squared" : "logistic"; \ 1; $$swap = 2 ** (int (4 * rand ())); \ 1; $$quad = rand () > 0.5 ? "z" : "b"; \ 1; $$cubic = rand () > 0.5 ? "z" : "b"; \ 1; print join " ", $$eta, $$nodes, $$loss, $$swap, $$quad, $$cubic;' | \ while read eta nodes loss swap quad cubic; \ do ( \ echo $$eta $$nodes $$loss $$swap $$quad $$cubic; \ time vw -f $@ --log_multi $$(( $(numlabels)+1 )) $< -q c$$quad --cubic cd$$cubic -b $(bits) --loss_function $$loss -l $$eta --swap_resistance $$swap; \ ) > full_log_multihyper$*.train 2>&1; \ done full_log_multihyper%: full_log_multihyper%.vw enwik9.test.vw.gz (time vw -t -i $^) > $@ 2>&1 log_multihyper%: bits=30 log_multihyper%: enwik8.train.vw.gz perl -le 'BEGIN { srand $*; }; \ 1; $$eta = 0.1 + 3.9 * rand (); \ 1; $$nodes = int (10000 + 40000 * rand ()); \ 1; $$loss = rand () > 0.5 ? "squared" : "logistic"; \ 1; $$swap = 2 ** (int (4 * rand ())); \ 1; $$quad = rand () > 0.5 ? "z" : "b"; \ 1; $$cubic = rand () > 0.5 ? "z" : "b"; \ 1; print join " ", $$eta, $$nodes, $$loss, $$swap, $$quad, $$cubic;' | \ while read eta nodes loss swap quad cubic; \ do ( \ echo $$eta $$nodes $$loss $$swap $$quad $$cubic; \ vw --log_multi $$(( $(numlabels)+1 )) $< -q c$$quad --cubic cd$$cubic -b $(bits) --loss_function $$loss -l $$eta --swap_resistance $$swap; \ ) > $@ 2>&1; \ done full_inorder_hyper%.vw: enwik9.inorder.train.vw.gz perl -le 'BEGIN { srand $*; }; \ 1; $$eta = 0.1 + 3.9 * rand (); \ 1; $$depth = int (11 + 6 * rand ()); \ 1; $$mc = int (20 + 80 * rand ()); \ 1; $$log_hyper = log (0.01) + (log (10) - log (0.01)) * rand (); \ 1; $$rr = rand () > 0.5 ? 1 : 0; \ 1; $$no = rand () > 0.5 ? 1 : 0; \ 1; $$dq = rand () > 0.5 ? "z" : "b"; \ 1; $$cubic = rand () > 0.5 ? "z" : "b"; \ 1; $$pf = rand () > 0.5 ? "z" : "b"; \ 1; $$pfc = rand () > 0.5 ? "z" : "b"; \ 1; print join " ", $$eta, $$depth, $$mc, exp ($$log_hyper), $$rr, $$no, $$dq, $$cubic, $$pf, $$pfc;' | \ while read eta depth mc hyper rr no dq cubic pf pfc; \ do ( \ echo $$eta $$depth $$mc $$hyper $$rr $$no $$dq $$cubic $$pf $$pfc; \ time vw -f $@ $< -b 32 --recall_tree $$(( $(numlabels)+1 )) -l $$eta --max_depth $$depth --max_candidates $$mc --bern_hyper $$hyper --randomized_routing $$rr --node_only $$no -q c$$dq --cubic cd$$cubic -q '\x88'$$pf --cubic '\x88'c$$pfc; \ ) > full_inorder_hyper$*.train 2>&1; \ done full_hyper%: full_hyper%.vw enwik9.test.vw.gz (time vw -t -i $^) > $@ 2>&1 full_hyper%.vw: enwik9.train.vw.gz perl -le 'BEGIN { srand $*; }; \ 1; $$eta = 0.1 + 3.9 * rand (); \ 1; $$depth = int (11 + 6 * rand ()); \ 1; $$mc = int (20 + 80 * rand ()); \ 1; $$log_hyper = log (0.01) + (log (10) - log (0.01)) * rand (); \ 1; $$rr = rand () > 0.5 ? 1 : 0; \ 1; $$no = rand () > 0.5 ? 1 : 0; \ 1; $$dq = rand () > 0.5 ? "z" : "b"; \ 1; $$cubic = rand () > 0.5 ? "z" : "b"; \ 1; $$pf = rand () > 0.5 ? "z" : "b"; \ 1; $$pfc = rand () > 0.5 ? "z" : "b"; \ 1; print join " ", $$eta, $$depth, $$mc, exp ($$log_hyper), $$rr, $$no, $$dq, $$cubic, $$pf, $$pfc;' | \ while read eta depth mc hyper rr no dq cubic pf pfc; \ do ( \ echo $$eta $$depth $$mc $$hyper $$rr $$no $$dq $$cubic $$pf $$pfc; \ time vw -f $@ $< -b 32 --recall_tree $$(( $(numlabels)+1 )) -l $$eta --max_depth $$depth --max_candidates $$mc --bern_hyper $$hyper --randomized_routing $$rr --node_only $$no -q c$$dq --cubic cd$$cubic -q '\x88'$$pf --cubic '\x88'c$$pfc; \ ) > full_hyper$*.train 2>&1; \ done full_oaa_hyper%: full_oaa_hyper%.vw enwik9.test.vw.gz (time vw -t -i $^) > $@ 2>&1 # hogwild fails with more bits ... some kind of deadlock issue :( full_oaa_hyper%.vw: bits=30 full_oaa_hyper%.vw: enwik9.train.vw.gz perl -le 'BEGIN { srand $*; }; \ 1; $$eta = 0.1 + 3.9 * rand (); \ 1; print join " ", $$eta;' | \ while read eta; \ do ( \ echo $$eta; \ unbuffer ./do-oaa-hogwild $< -f $@ --oaa $$(( $(numlabels)+1 )) --oaa_subsample 2000 -q bc --cubic bcd -b $(bits) --loss_function logistic -l $$eta; \ ) |& > full_oaa_hyper$*.train 2>&1; \ done oaa_hyper%: bits=30 oaa_hyper%: enwik8.train.vw.gz perl -le 'BEGIN { srand $*; }; \ 1; $$eta = 0.1 + 3.9 * rand (); \ 1; print join " ", $$eta;' | \ while read eta; \ do ( \ echo $$eta; \ vw $< --oaa $$(( $(numlabels)+1 )) --oaa_subsample 2000 -q bc --cubic bcd -b $(bits) --loss_function logistic -l $$eta; \ ) |& > $@ 2>&1; \ done hyper%: enwik8.train.vw.gz perl -le 'BEGIN { srand $*; }; \ 1; $$eta = 0.1 + 3.9 * rand (); \ 1; $$depth = int (11 + 6 * rand ()); \ 1; $$mc = int (20 + 80 * rand ()); \ 1; $$log_hyper = log (0.01) + (log (10) - log (0.01)) * rand (); \ 1; $$rr = rand () > 0.5 ? 1 : 0; \ 1; $$no = rand () > 0.5 ? 1 : 0; \ 1; $$dq = rand () > 0.5 ? "z" : "b"; \ 1; $$cubic = rand () > 0.5 ? "z" : "b"; \ 1; $$pf = rand () > 0.5 ? "z" : "b"; \ 1; $$pfc = rand () > 0.5 ? "z" : "b"; \ 1; print join " ", $$eta, $$depth, $$mc, exp ($$log_hyper), $$rr, $$no, $$dq, $$cubic, $$pf, $$pfc;' | \ while read eta depth mc hyper rr no dq cubic pf pfc; \ do ( \ echo $$eta $$depth $$mc $$hyper $$rr $$no $$dq $$cubic $$pf $$pfc; \ vw $< -b 30 --recall_tree $$(( $(numlabels)+1 )) -l $$eta --max_depth $$depth --max_candidates $$mc --bern_hyper $$hyper --randomized_routing $$rr --node_only $$no -q c$$dq --cubic cd$$cubic -q '\x88'$$pf --cubic '\x88'c$$pfc; \ ) > $@ 2>&1; \ done sweep_recall_tree%: sweep_recall_tree%.vw enwik8.test.vw.gz (time vw -t -i $^) > $@ 2>&1 sweep_recall_tree%.vw: bern_hyper=1 sweep_recall_tree%.vw: enwik8.train.vw.gz (time vw -f $@ --recall_tree $$(( $(numlabels)+1 )) $< -q bc --cubic bcd -b 30 --loss_function logistic -l 1 -q '\x88'b --cubic '\x88'bc --max_candidates $* --bern_hyper $(bern_hyper)) > sweep_recall_tree$*.train 2>&1 recall_tree%: recall_tree%.vw enwik%.test.vw.gz vw -t -i $^ recall_tree%.vw: vwextra= recall_tree%.vw: bits=32 recall_tree%.vw: enwik%.train.vw.gz vw -f $@ --recall_tree $$(( $(numlabels)+1 )) $< -q bc --cubic bcd -b $(bits) --loss_function logistic -l 1 -q '\x88'b --cubic '\x88'bc $(vwextra) #--link glf1 --randomized_routing $(randomized_routing) log_multi%: log_multi%.vw enwik%.test.vw.gz time vw -t -i $^ log_multi%.vw: enwik%.train.vw.gz time vw -f $@ --log_multi $$(( $(numlabels)+1 )) $< -q bc --cubic bcd -b 32 --loss_function logistic # NB: 30 instead of 32 here ... hogwild increases memory usage (?) oaahogwild%.vw: enwik%.train.vw.gz ./do-oaa-hogwild $< -f $@ --oaa $$(( $(numlabels)+1 )) --oaa_subsample 2000 -q bc --cubic bcd -b 30 --loss_function logistic -l 1 oaahogwild%: enwik%.test.vw.gz oaahogwild%.vw ./do-oaa-hogwild $< -i $(word 2,$^) -t enwik%.zip: wget -c http://mattmahoney.net/dc/enwik$*.zip enwik%.freq: enwik%.zip zcat enwik$*.zip | \ perl ./wikifil.pl | \ perl -lane 'foreach (@F) { $$c{$$_}++; } } \ 1; { while (($$k, $$v) = each %c) { print "$$k\t$$v"; }' | \ sort -k2rn > $@ enwik%.train.vw.gz: enwik%.freq enwik%.zip zcat enwik$*.zip | \ perl ./wikifil.pl | \ perl -ne 'BEGIN { srand 69; }; \ 1; if (10 * rand () < 1) { print STDERR $$_; \ 1; } else { print STDOUT $$_; }' \ 2> >(perl ./gendata.pl enwik$*.freq $(numlabels) 10000000 | gzip > enwik$*.test.vw.gz) \ > >(perl ./gendata.pl enwik$*.freq $(numlabels) 10000000 | gzip > enwik$*.train.vw.gz) # NB: shufbuf size of 1 => in order processing enwik%.inorder.train.vw.gz: enwik%.freq enwik%.zip zcat enwik$*.zip | \ perl ./wikifil.pl | \ perl -ne 'BEGIN { srand 69; }; \ 1; if (10 * rand () < 1) { print STDERR $$_; \ 1; } else { print STDOUT $$_; }' \ 2> >(perl ./gendata.pl enwik$*.freq $(numlabels) 1 | gzip > enwik$*.inorder.test.vw.gz) \ > >(perl ./gendata.pl enwik$*.freq $(numlabels) 1 | gzip > enwik$*.inorder.train.vw.gz) vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/ltcb/README.md000066400000000000000000000017321323713302100227130ustar00rootroot00000000000000Recall tree demo for ltcb ------------------------------- There are two versions of the LTCB: enwik8 is the small one, and enwik9 is the big one. http://mattmahoney.net/dc/textdata.html * make enwik8.train.vw.gz: makes the following files 1. enwik8.zip: downloaded 2. enwik8.freq: base frequency of each token in descending order 2a. labels are the first 80K of these 3. enwik8.train.vw.gz: training set 4. enwik8.test.vw.gz: test set * make oaa8: example invocation of VW on oaa for enwik8 * make enwik9.train.vw.gz: same files but larger (number of examples) * make oaa9: example invocation of VW on oaa for enwik9 the VW data files consist of the label and the 10 preceeding words, no boundary modeling, i.e., 11th word is first example generated, and there is no attempt to predict ``end of document'' infrequent words are replaced with unknown in the labels, so label 80001 is the unknown label. however all words are preseved in input. vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/ltcb/do-oaa-hogwild000077500000000000000000000040251323713302100241530ustar00rootroot00000000000000#! /bin/zsh nukeem() { \ trap - INT QUIT TERM pkill -9 -f 'vw.*--port 26542' } if which netcat >/dev/null 2>/dev/null; then netcat=netcat elif which nc >/dev/null 2>/dev/null; then netcat=nc else echo "Install netcat or nc" >&2 exit 1 fi data="$1" shift { vw "$@" --num_children 24 --port 26542 2>&1 | perl -lane 'print $_ unless $F[2] =~ /^\d$/ && $c{$F[2]}++;' } & trap 'nukeem; exit 1' INT QUIT TERM while ! $netcat -z localhost 26542 do sleep 1 done zcat $data | \ ./map \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) \ >($netcat localhost 26542 > /dev/null) pkill -f 'vw.*--port 26542' while test "x$1" = "x-f" && test ! -s "$2" do sleep 1 done vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/ltcb/gendata.pl000077500000000000000000000021761323713302100234020ustar00rootroot00000000000000#! /usr/bin/env perl use warnings; use strict; use IO::File; my $freqfile = shift @ARGV or die; my $numlabels = shift @ARGV or die; my $shufbufsize = shift @ARGV or die; my $freqfh = new IO::File $freqfile, "r" or die "$freqfile: $!"; my %dict; my $curlabels = $numlabels; while (defined ($_ = <$freqfh>)) { chomp; my ($key, undef) = split /\s+/, $_; $dict{$key} = $curlabels; --$curlabels; last unless $curlabels > 0; } srand 69; my $context = 6; my @shufbuf; my @charpos = split //, "abcdefghijklmnop"; $\="\n"; while (defined ($_ = )) { chomp; s/\|/_/g; s/:/_/g; # VW special characters my @tokens = split /\s+/, $_; foreach my $pos ($context ... $#tokens) { my $label = $dict{$tokens[$pos]} || $numlabels+1; my $index = int (rand ($shufbufsize)); print $shufbuf[$index] if length ($shufbuf[$index]); $shufbuf[$index] = join " ", $label, map { join "", " |", $charpos[$_], " ", $tokens[$pos - $_] } (1 .. $context); } } foreach my $index (0 .. $shufbufsize) { print $shufbuf[$index] if length ($shufbuf[$index]); } vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/ltcb/map000077500000000000000000000010551323713302100221350ustar00rootroot00000000000000#! /usr/bin/env perl use warnings; use strict; use IO::File; use IO::Select; my @mappers = map { $_->autoflush (0); $_ } map { new IO::File $_, "w" or die "$_: $!" } @ARGV; my $bufsize = 6; my @buf = grep { defined } map { scalar } (1 .. $bufsize); my $n = 0; while (@buf) { my $fh = $mappers[$n % @mappers]; my @ready = IO::Select->new ($fh)->can_write (100); if (@ready) { print $fh @buf; @buf = grep { defined } map { scalar } (1 .. $bufsize); } ++$n; } vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/ltcb/wikifil.pl000077500000000000000000000040651323713302100234340ustar00rootroot00000000000000#!/usr/bin/perl # Program to filter Wikipedia XML dumps to "clean" text consisting only of lowercase # letters (a-z, converted from A-Z), and spaces (never consecutive). # All other characters are converted to spaces. Only text which normally appears # in the web browser is displayed. Tables are removed. Image captions are # preserved. Links are converted to normal text. Digits are spelled out. # Written by Matt Mahoney, June 10, 2006. This program is released to the public domain. $/=">"; # input record separator $\="\n"; # output record separator while (<>) { if (/ ... if (/#redirect/i) {$text=0;} # remove #REDIRECT if ($text) { # Remove any text not normally visible if (/<\/text>/) {$text=0;} s/\n/ /g; # remove newlines s/<.*>//; # remove xml tags s/&/&/g; # decode URL encoded chars s/<//g; s///g; # remove references ... s/<[^>]*>//g; # remove xhtml tags s/\[http:[^] ]*/[/g; # remove normal url, preserve visible text s/\|thumb//ig; # remove images links, preserve caption s/\|left//ig; s/\|right//ig; s/\|\d+px//ig; s/\[\[image:[^\[\]]*\|//ig; s/\[\[category:([^|\]]*)[^]]*\]\]/[[$1]]/ig; # show categories without markup s/\[\[[a-z\-]*:[^\]]*\]\]//g; # remove links to other languages s/\[\[[^\|\]]*\|/[[/g; # remove wiki url, preserve visible text s/{{[^}]*}}//g; # remove {{icons}} and {tables} s/{[^}]*}//g; s/\[//g; # remove [ and ] s/\]//g; s/&[^;]*;/ /g; # remove URL encoded chars # convert to lowercase letters and spaces, spell digits $_=" $_ "; tr/A-Z/a-z/; s/0/ zero /g; s/1/ one /g; s/2/ two /g; s/3/ three /g; s/4/ four /g; s/5/ five /g; s/6/ six /g; s/7/ seven /g; s/8/ eight /g; s/9/ nine /g; tr/a-z/ /cs; chop; print $_ if $_ =~ /\S/; } } vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/odp/000077500000000000000000000000001323713302100212675ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/odp/Makefile000066400000000000000000000134471323713302100227400ustar00rootroot00000000000000SHELL=/bin/zsh .SECONDARY: numlabels=105033 bits?=31 odp_train.vw.gz: wget --tries=100 -c http://hunch.net/~vw/odp_train.vw.gz odp_test.vw.gz: wget --tries=100 -c http://hunch.net/~vw/odp_test.vw.gz full_hyper%: full_hyper%.vw odp_test.vw.gz time vw -t -i $(word 1,$^) <(zcat $(word 2,$^) | sed 's/|/|b/') ufull_hyper%.vw: bits=30 ufull_hyper%.vw: odp_train.vw.gz perl -le 'BEGIN { srand $*; }; \ 1; $$eta = 0.1 + 3.9 * rand (); \ 1; $$depth = int (12 + 6 * rand ()); \ 1; $$mc = int (40 + 80 * rand ()); \ 1; $$log_hyper = log (0.01) + (log (10) - log (0.01)) * rand (); \ 1; $$rr = rand () > 0.5 ? 1 : 0; \ 1; $$dq = rand () > 0.5 ? "a" : "b"; \ 1; $$no = rand () > 0.5 ? "0" : "1"; \ 1; print join " ", $$eta, $$depth, $$mc, exp ($$log_hyper), $$rr, $$dq, $$no;' | \ while read eta depth mc hyper rr dq no; \ do ( \ echo $$eta $$depth $$mc $$hyper $$rr $$dq $$no; \ time vw -f $@ --recall_tree $(numlabels) <((zcat $<; zcat $<; zcat $<; zcat $<; zcat $<) | sed 's/|/|b/' | ./minishuf 100000) -b $(bits) --loss_function logistic -l $$eta --max_depth $$depth --max_candidates $$mc --bern_hyper $$hyper --link glf1 --randomized_routing $$rr --node_only $$no -q '\x88'$$dq; \ ) > $@.train 2>&1; \ done pfull_hyper%.vw: bits=30 pfull_hyper%.vw: odp_train.vw.gz perl -le 'BEGIN { srand $*; }; \ 1; $$eta = 0.1 + 3.9 * rand (); \ 1; $$depth = int (12 + 6 * rand ()); \ 1; $$mc = int (40 + 80 * rand ()); \ 1; $$log_hyper = log (0.01) + (log (10) - log (0.01)) * rand (); \ 1; $$rr = rand () > 0.5 ? 1 : 0; \ 1; $$dq = rand () > 0.5 ? "a" : "b"; \ 1; $$no = rand () > 0.5 ? "0" : "1"; \ 1; print join " ", $$eta, $$depth, $$mc, exp ($$log_hyper), $$rr, $$dq, $$no;' | \ while read eta depth mc hyper rr dq no; \ do ( \ echo $$eta $$depth $$mc $$hyper $$rr $$dq $$no; \ time vw -f $@ --recall_tree $(numlabels) <((zcat $<; zcat $<; zcat $<; zcat $<) | sed 's/|/|b/' | ./minishuf 100000) -b $(bits) --loss_function logistic -l $$eta --max_depth $$depth --max_candidates $$mc --bern_hyper $$hyper --link glf1 --randomized_routing $$rr --node_only $$no -q '\x88'$$dq; \ ) > $@.train 2>&1; \ done mfull_hyper%.vw: bits=30 mfull_hyper%.vw: odp_train.vw.gz perl -le 'BEGIN { srand $*; }; \ 1; $$eta = 0.1 + 3.9 * rand (); \ 1; $$depth = int (12 + 6 * rand ()); \ 1; $$mc = int (40 + 80 * rand ()); \ 1; $$log_hyper = log (0.01) + (log (10) - log (0.01)) * rand (); \ 1; $$rr = rand () > 0.5 ? 1 : 0; \ 1; $$dq = rand () > 0.5 ? "a" : "b"; \ 1; $$no = rand () > 0.5 ? "0" : "1"; \ 1; print join " ", $$eta, $$depth, $$mc, exp ($$log_hyper), $$rr, $$dq, $$no;' | \ while read eta depth mc hyper rr dq no; \ do ( \ echo $$eta $$depth $$mc $$hyper $$rr $$dq $$no; \ time vw -f $@ --recall_tree $(numlabels) <((zcat $<; zcat $<; zcat $<) | sed 's/|/|b/' | ./minishuf 100000) -b $(bits) --loss_function logistic -l $$eta --max_depth $$depth --max_candidates $$mc --bern_hyper $$hyper --link glf1 --randomized_routing $$rr --node_only $$no -q '\x88'$$dq; \ ) > $@.train 2>&1; \ done full_hyper%.vw: bits=31 full_hyper%.vw: odp_train.vw.gz perl -le 'BEGIN { srand $*; }; \ 1; $$eta = 0.1 + 3.9 * rand (); \ 1; $$depth = int (12 + 6 * rand ()); \ 1; $$mc = int (40 + 80 * rand ()); \ 1; $$log_hyper = log (0.01) + (log (10) - log (0.01)) * rand (); \ 1; $$rr = rand () > 0.5 ? 1 : 0; \ 1; $$dq = rand () > 0.5 ? "a" : "b"; \ 1; $$no = rand () > 0.5 ? "0" : "1"; \ 1; print join " ", $$eta, $$depth, $$mc, exp ($$log_hyper), $$rr, $$dq, $$no;' | \ while read eta depth mc hyper rr dq no; \ do ( \ echo $$eta $$depth $$mc $$hyper $$rr $$dq $$no; \ time vw -f $@ --recall_tree $(numlabels) <((zcat $<; zcat $<) | sed 's/|/|b/' | ./minishuf 100000) -b $(bits) --loss_function logistic -l $$eta --max_depth $$depth --max_candidates $$mc --bern_hyper $$hyper --link glf1 --randomized_routing $$rr --node_only $$no -q '\x88'$$dq; \ ) > full_hyper$*.train 2>&1; \ done hyper%: bits=29 hyper%: odp_train.vw.gz perl -le 'BEGIN { srand $*; }; \ 1; $$eta = 0.1 + 3.9 * rand (); \ 1; $$depth = int (8 + 6 * rand ()); \ 1; $$mc = int (40 + 120 * rand ()); \ 1; $$log_hyper = log (0.01) + (log (10) - log (0.01)) * rand (); \ 1; $$rr = rand () > 0.5 ? 1 : 0; \ 1; $$dq = rand () > 0.5 ? "a" : "b"; \ 1; $$no = rand () > 0.5 ? "0" : "1"; \ 1; print join " ", $$eta, $$depth, $$mc, exp ($$log_hyper), $$rr, $$dq, $$no;' | \ while read eta depth mc hyper rr dq no; \ do ( \ echo $$eta $$depth $$mc $$hyper $$rr $$dq $$no; \ vw --recall_tree $(numlabels) <(zcat $< | sed 's/|/|b/' | ./minishuf 100000) -b $(bits) --loss_function logistic -l $$eta --max_depth $$depth --max_candidates $$mc --bern_hyper $$hyper --link glf1 --randomized_routing $$rr --node_only $$no -q '\x88'$$dq; \ ) > $@ 2>&1; \ done log_multi: log_multi.vw odp_test.vw.gz time vw -t -i $^ log_multi.vw: odp_train.vw.gz time vw -f $@ --log_multi $(numlabels) $< -b $(bits) --loss_function logistic -l 1 recall_tree: recall_tree.vw odp_test.vw.gz time vw -t -i $(word 1,$^) <(zcat $(word 2,$^) | sed 's/|/|b/') recall_tree.vw: odp_train.vw.gz time vw -f $@ --recall_tree $(numlabels) <(zcat $< | sed 's/|/|b/' | ./minishuf 100000) -b $(bits) --loss_function logistic -l 1 --node_only 1 -q '\x88'b --max_candidates 400 --max_depth 5 oaahogwild.vw: odp_train.vw.gz time ./do-oaa-hogwild <(zcat $< | ./minishuf 100000 | gzip -c) -f $@ --oaa $(numlabels) --oaa_subsample 1000 -b $(bits) --loss_function logistic -l 1 oaahogwild: oaahogwild.vw odp_test.vw.gz time ./do-oaa-hogwild $(word 2,$^) -t -i $(word 1,$^) vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/odp/README.md000066400000000000000000000003651323713302100225520ustar00rootroot00000000000000Recall tree demo for odp ------------------------------- This demo exercises the recall tree reduction for logarithmic time multiclass classification on the odp dataset. It takes a long time to run, a lot of memory, and a lot of disk space. vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/odp/do-oaa-hogwild000077500000000000000000000040371323713302100240140ustar00rootroot00000000000000#! /bin/zsh nukeem() { \ trap - INT QUIT TERM pkill -9 -f 'vw.*--port 26544' } if which netcat >/dev/null 2>/dev/null; then netcat=netcat elif which nc >/dev/null 2>/dev/null; then netcat=nc else echo "Install netcat or nc" >&2 exit 1 fi data="$1" shift { vw "$@" --num_children 24 --port 26544 2>&1 | perl -lane 'print $_ unless $F[2] =~ /^\d$/ && $c{$F[2]}++;' } & trap 'nukeem; exit 1' INT QUIT TERM while ! $netcat -z localhost 26544 do sleep 1 done sleep 30 zcat $data | \ ./map \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) \ >($netcat localhost 26544 > /dev/null) pkill -f 'vw.*--port 26544' while test "x$1" = "x-f" && test ! -s "$2" do sleep 1 done vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/odp/map000077500000000000000000000010571323713302100217750ustar00rootroot00000000000000#! /usr/bin/env perl use warnings; use strict; use IO::File; use IO::Select; my @mappers = map { $_->autoflush (1); $_ } map { new IO::File $_, "w" or die "$_: $!" } @ARGV; my $bufsize = 4; my @buf = grep { defined } map { scalar } (1 .. $bufsize); my $n = 0; while (@buf) { my $fh = $mappers[$n % @mappers]; my @ready = IO::Select->new ($fh)->can_write (20000); if (@ready) { print $fh @buf; @buf = grep { defined } map { scalar } (1 .. $bufsize); } ++$n; } vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/odp/minishuf000077500000000000000000000005151323713302100230400ustar00rootroot00000000000000#! /usr/bin/env perl use warnings; use strict; srand 69; my $bufsize = shift @ARGV or die; my @buf; while (defined ($_ = <>)) { my $index = int rand $bufsize; print $buf[$index] if defined $buf[$index]; $buf[$index] = $_; } foreach my $index (0 .. $#buf) { print $buf[$index] if defined $buf[$index]; } vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/wikipara/000077500000000000000000000000001323713302100223145ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/wikipara/.gitignore000066400000000000000000000001211323713302100242760ustar00rootroot00000000000000docid2label enwiki-20170601-pages-articles-multistream.xml.bz2 paradata*.gz text vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/wikipara/DocGenerator.py000066400000000000000000000014541323713302100252460ustar00rootroot00000000000000import bz2 import re import string #------------------------------------------------- # iterate over documents #------------------------------------------------- def docs(filename): docid=None startdocregex=re.compile('": yield int(docid), paragraphs docid=None elif not line.isspace(): if "\n" in line: curpara.append(line.rstrip('\n')) paragraphs.append(' '.join(curpara)) curpara=[] else: curpara.appand(line) if docid is None: m=startdocregex.match (line) if m is not None: docid=m.group(1) paragraphs=[] curpara=[] vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/wikipara/Makefile000066400000000000000000000024511323713302100237560ustar00rootroot00000000000000SHELL=/bin/zsh .SECONDARY: .PHONY: all ratelimit?=1M all: paradata10000.vw.train.gz paradata%.vw.train.gz: makeparadata.py text/AA/wiki_00.shuf.bz2 python ./makeparadata.py $* 4 \ > >(perl -pe 'BEGIN { srand 69; }; $$_ = rand()."\t$$_";' | \ sort -t$$'\t' -k1n -S30% | cut -f2- | gzip > paradata$*.vw.train.gz) \ 2> >(perl -pe 'BEGIN { srand 69; }; $$_ = rand()."\t$$_";' | \ sort -t$$'\t' -k1n -S30% | cut -f2- | gzip > paradata$*.vw.test.gz) while ! test -s paradata$*.vw.test.gz; do sleep 1; done while ! test -s paradata$*.vw.train.gz; do sleep 1; done enwiki-20170601-pages-articles-multistream.xml.bz2: wget --limit-rate=$(ratelimit) http://dumps.wikimedia.your.org/enwiki/20170601/$@ text/AA/wiki_00.bz2: WikiExtractor.py enwiki-20170601-pages-articles-multistream.xml.bz2 python ./WikiExtractor.py --no-templates --bytes 100G -c <(bzcat enwiki-20170601-pages-articles-multistream.xml.bz2) text/AA/wiki_00.shuf.bz2: text/AA/wiki_00.bz2 bzcat $< | \ perl -ne 'BEGIN { srand 69; $$/="\n"; }; \ 1; $$r = rand (); print "$$r\001$$_\000";' | \ sort -k1n -t$$'\001' -z -S50% --compress-program=lzop | \ perl -F'\001' -ane 'BEGIN { $$/="\000"; }; \ 1; chomp($$F[1]); print $$F[1]' | \ bzip2 > $@ vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/wikipara/README.md000066400000000000000000000010331323713302100235700ustar00rootroot00000000000000Wikipara data set ------------------------------- This is a data set generation scheme we were playing around with which allows us to vary the number of classes and the number of training examples per class. We take some number of wikipedia pages and use the page (id) as the class. We then take some number of paragraphs from the page as training examples and some other number of paragraphs from the page as test examples. A setting we found particularly challenging was 10,000 classes with 3 training examples and 1 test example. vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/wikipara/WikiExtractor.py000077500000000000000000002672021323713302100255010ustar00rootroot00000000000000#!/usr/bin/python # -*- coding: utf-8 -*- # # ============================================================================= # Version: 2.39 (September 29, 2015) # Author: Giuseppe Attardi (attardi@di.unipi.it), University of Pisa # # Contributors: # Antonio Fuschetto (fuschett@aol.com) # Leonardo Souza (lsouza@amtera.com.br) # Juan Manuel Caicedo (juan@cavorite.com) # Humberto Pereira (begini@gmail.com) # Siegfried-A. Gevatter (siegfried@gevatter.com) # Pedro Assis (pedroh2306@gmail.com) # Wim Muskee (wimmuskee@gmail.com) # Radics Geza (radicsge@gmail.com) # # ============================================================================= # Copyright (c) 2009-2015. Giuseppe Attardi (attardi@di.unipi.it). # ============================================================================= # This file is part of Tanl. # # Tanl is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License, version 3, # as published by the Free Software Foundation. # # Tanl is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . # ============================================================================= """Wikipedia Extractor: Extracts and cleans text from a Wikipedia database dump and stores output in a number of files of similar size in a given directory. Each file will contain several documents in the format: ... This version performs template expansion by preprocesssng the whole dump and collecting template definitions. """ import sys, os.path, time import re # TODO use regex when it will be standard import argparse, random from itertools import izip, izip_longest import logging import urllib import bz2 import codecs from htmlentitydefs import name2codepoint from multiprocessing import Queue, JoinableQueue, Process, Manager, cpu_count from cStringIO import StringIO import fileinput from timeit import default_timer #=========================================================================== # Program version version = '2.39' ### PARAMS #################################################################### ## # Defined in # We include as default Template, when loading external template file. knownNamespaces = set(['Template']) ## # The namespace used for template definitions # It is the name associated with namespace key=10 in the siteinfo header. templateNamespace = '' ## # The namespace used for module definitions # It is the name associated with namespace key=828 in the siteinfo header. moduleNamespace = '' ## # Recognize only these namespaces # w: Internal links to the Wikipedia # wiktionary: Wiki dictionary # wikt: shortcut for Wiktionary # acceptedNamespaces = ['w', 'wiktionary', 'wikt'] ## # Drop these elements from article text # discardElements = [ 'gallery', 'timeline', 'noinclude', 'pre', 'table', 'tr', 'td', 'th', 'caption', 'div', 'form', 'input', 'select', 'option', 'textarea', 'ul', 'li', 'ol', 'dl', 'dt', 'dd', 'menu', 'dir', 'ref', 'references', 'img', 'imagemap', 'source', 'small' ] # This is obtained from urlbase = None def get_url(id): global urlbase return "%s?curid=%s" % (urlbase, id) #========================================================================= # # MediaWiki Markup Grammar # https://www.mediawiki.org/wiki/Preprocessor_ABNF # xml-char = %x9 / %xA / %xD / %x20-D7FF / %xE000-FFFD / %x10000-10FFFF # sptab = SP / HTAB # ; everything except ">" (%x3E) # attr-char = %x9 / %xA / %xD / %x20-3D / %x3F-D7FF / %xE000-FFFD / %x10000-10FFFF # literal = *xml-char # title = wikitext-L3 # part-name = wikitext-L3 # part-value = wikitext-L3 # part = ( part-name "=" part-value ) / ( part-value ) # parts = [ title *( "|" part ) ] # tplarg = "{{{" parts "}}}" # template = "{{" parts "}}" # link = "[[" wikitext-L3 "]]" # comment = "" # unclosed-comment = "', re.DOTALL) # Match ignored tags ignored_tag_patterns = [] def ignoreTag(tag): left = re.compile(r'<%s\b.*?>' % tag, re.IGNORECASE | re.DOTALL) # both and right = re.compile(r'' % tag, re.IGNORECASE) ignored_tag_patterns.append((left, right)) for tag in ignoredTags: ignoreTag(tag) # Match selfClosing HTML tags selfClosing_tag_patterns = [ re.compile(r'<\s*%s\b[^>]*/\s*>' % tag, re.DOTALL | re.IGNORECASE) for tag in selfClosingTags ] # Match HTML placeholder tags placeholder_tag_patterns = [ (re.compile(r'<\s*%s(\s*| [^>]+?)>.*?<\s*/\s*%s\s*>' % (tag, tag), re.DOTALL | re.IGNORECASE), repl) for tag, repl in placeholder_tags.items() ] # Match preformatted lines preformatted = re.compile(r'^ .*?$') # Match external links (space separates second optional parameter) externalLink = re.compile(r'\[\w+[^ ]*? (.*?)]') externalLinkNoAnchor = re.compile(r'\[\w+[&\]]*\]') # Matches bold/italic bold_italic = re.compile(r"'''''(.*?)'''''") bold = re.compile(r"'''(.*?)'''") italic_quote = re.compile(r"''\"([^\"]*?)\"''") italic = re.compile(r"''(.*?)''") quote_quote = re.compile(r'""([^"]*?)""') # Matches space spaces = re.compile(r' {2,}') # Matches dots dots = re.compile(r'\.{4,}') #====================================================================== class Template(list): """ A Template is a list of TemplateText or TemplateArgs """ @classmethod def parse(cls, body): tpl = Template() # we must handle nesting, s.a. # {{{1|{{PAGENAME}}} # {{{italics|{{{italic|}}} # {{#if:{{{{{#if:{{{nominee|}}}|nominee|candidate}}|}}}| # start = 0 for s,e in findMatchingBraces(body, 3): tpl.append(TemplateText(body[start:s])) tpl.append(TemplateArg(body[s+3:e-3])) start = e tpl.append(TemplateText(body[start:])) # leftover return tpl def subst(self, params, extractor, depth=0): # We perform parameter substitutions recursively. # We also limit the maximum number of iterations to avoid too long or # even endless loops (in case of malformed input). # :see: http://meta.wikimedia.org/wiki/Help:Expansion#Distinction_between_variables.2C_parser_functions.2C_and_templates # # Parameter values are assigned to parameters in two (?) passes. # Therefore a parameter name in a template can depend on the value of # another parameter of the same template, regardless of the order in # which they are specified in the template call, for example, using # Template:ppp containing "{{{{{{p}}}}}}", {{ppp|p=q|q=r}} and even # {{ppp|q=r|p=q}} gives r, but using Template:tvvv containing # "{{{{{{{{{p}}}}}}}}}", {{tvvv|p=q|q=r|r=s}} gives s. #logging.debug('subst tpl (%d, %d) %s', len(extractor.frame), depth, self) if depth > extractor.maxParameterRecursionLevels: extractor.recursion_exceeded_3_errs += 1 return '' return ''.join([tpl.subst(params, extractor, depth) for tpl in self]) def __str__(self): return ''.join([unicode(x) for x in self]) class TemplateText(unicode): """Fixed text of template""" def subst(self, params, extractor, depth): return self class TemplateArg(object): """ parameter to a template. Has a name and a default value, both of which are Templates. """ def __init__(self, parameter): """ :param parameter: the parts of a tplarg. """ # the parameter name itself might contain templates, e.g.: # appointe{{#if:{{{appointer14|}}}|r|d}}14| # 4|{{{{{subst|}}}CURRENTYEAR}} # any parts in a tplarg after the first (the parameter default) are # ignored, and an equals sign in the first part is treated as plain text. #logging.debug('TemplateArg %s', parameter) parts = splitParts(parameter) self.name = Template.parse(parts[0]) if len(parts) > 1: # This parameter has a default value self.default = Template.parse(parts[1]) else: self.default = None def __str__(self): if self.default: return '{{{%s|%s}}}' % (self.name, self.default) else: return '{{{%s}}}' % self.name def subst(self, params, extractor, depth): """ Substitute value for this argument from dict :param params: Use :param extractor: to evaluate expressions for name and default. Limit substitution to the maximun :param depth:. """ # the parameter name itself might contain templates, e.g.: # appointe{{#if:{{{appointer14|}}}|r|d}}14| paramName = self.name.subst(params, extractor, depth+1) paramName = extractor.expandTemplates(paramName) res = '' if paramName in params: res = params[paramName] # use parameter value specified in template invocation elif self.default: # use the default value defaultValue = self.default.subst(params, extractor, depth+1) res = extractor.expandTemplates(defaultValue) #logging.debug('subst arg %d %s -> %s' % (depth, paramName, res)) return res #====================================================================== substWords = 'subst:|safesubst:' class Extractor(object): """ An extraction task on a article. """ ## # Whether to preserve links in output keepLinks = False ## # Whether to transform sections into HTML keepSections = False ## # Whether to output HTML instead of text toHTML = False def __init__(self, id, title, page): """ :param page: a list of lines. """ self.id = id self.title = title self.page = page self.magicWords = MagicWords() self.frame = [] self.recursion_exceeded_1_errs = 0 # template recursion within expandTemplates() self.recursion_exceeded_2_errs = 0 # template recursion within expandTemplate() self.recursion_exceeded_3_errs = 0 # parameter recursion self.template_title_errs = 0 def extract(self, out): """ :param out: a memory file. """ logging.debug("%s\t%s", self.id, self.title) text = ''.join(self.page) url = get_url(self.id) header = '\n' % (self.id, url, self.title) # Separate header from text with a newline. header += self.title + '\n\n' header = header.encode('utf-8') self.magicWords['pagename'] = self.title self.magicWords['fullpagename'] = self.title self.magicWords['currentyear'] = time.strftime('%Y') self.magicWords['currentmonth'] = time.strftime('%m') self.magicWords['currentday'] = time.strftime('%d') self.magicWords['currenthour'] = time.strftime('%H') self.magicWords['currenttime'] = time.strftime('%H:%M:%S') text = clean(self, text) footer = "\n\n" out.write(header) for line in compact(text): out.write(line.encode('utf-8')) out.write('\n') out.write(footer) errs = (self.template_title_errs, self.recursion_exceeded_1_errs, self.recursion_exceeded_2_errs, self.recursion_exceeded_3_errs) if any(errs): logging.warn("Template errors in article '%s' (%s): title(%d) recursion(%d, %d, %d)", self.title, self.id, *errs) #---------------------------------------------------------------------- # Expand templates maxTemplateRecursionLevels = 30 maxParameterRecursionLevels = 10 # check for template beginning reOpen = re.compile('(?= self.maxTemplateRecursionLevels: self.recursion_exceeded_1_errs += 1 return res #logging.debug(' %d %s', len(self.frame), res) return res def templateParams(self, parameters): """ Build a dictionary with positional or name key to expanded parameters. :param parameters: the parts[1:] of a template, i.e. all except the title. :param depth: recusion depth. """ templateParams = {} if not parameters: return templateParams logging.debug(' # Parameters may span several lines, like: # {{Reflist|colwidth=30em|refs= # <ref name="Goode">Title</ref> # The '=' might occurr within an HTML attribute: # "<ref name=value" # but we stop at first. m = re.match(' *([^=]*?) *=(.*)', param, re.DOTALL) if m: # This is a named parameter. This case also handles parameter # assignments like "2=xxx", where the number of an unnamed # parameter ("2") is specified explicitly - this is handled # transparently. parameterName = m.group(1).strip() parameterValue = m.group(2) if ']]' not in parameterValue: # if the value does not contain a link, trim whitespace parameterValue = parameterValue.strip() templateParams[parameterName] = parameterValue else: # this is an unnamed parameter unnamedParameterCounter += 1 if ']]' not in param: # if the value does not contain a link, trim whitespace param = param.strip() templateParams[str(unnamedParameterCounter)] = param logging.debug(' templateParams> %s', '|'.join(templateParams.values())) return templateParams def expandTemplate(self, body): """Expands template invocation. :param body: the parts of a template. :see http://meta.wikimedia.org/wiki/Help:Expansion for an explanation of the process. See in particular: Expansion of names and values http://meta.wikimedia.org/wiki/Help:Expansion#Expansion_of_names_and_values For most parser functions all names and values are expanded, regardless of what is relevant for the result. The branching functions (#if, #ifeq, #iferror, #ifexist, #ifexpr, #switch) are exceptions. All names in a template call are expanded, and the titles of the tplargs in the template body, after which it is determined which values must be expanded, and for which tplargs in the template body the first part (default). In the case of a tplarg, any parts beyond the first are never expanded. The possible name and the value of the first part is expanded if the title does not match a name in the template call. :see code for braceSubstitution at https://doc.wikimedia.org/mediawiki-core/master/php/html/Parser_8php_source.html#3397: """ # template = "{{" parts "}}" # Templates and tplargs are decomposed in the same way, with pipes as # separator, even though eventually any parts in a tplarg after the first # (the parameter default) are ignored, and an equals sign in the first # part is treated as plain text. # Pipes inside inner templates and tplargs, or inside double rectangular # brackets within the template or tplargs are not taken into account in # this decomposition. # The first part is called title, the other parts are simply called parts. # If a part has one or more equals signs in it, the first equals sign # determines the division into name = value. Equals signs inside inner # templates and tplargs, or inside double rectangular brackets within the # part are not taken into account in this decomposition. Parts without # equals sign are indexed 1, 2, .., given as attribute in the tag. if len(self.frame) >= self.maxTemplateRecursionLevels: self.recursion_exceeded_2_errs += 1 #logging.debug(' INVOCATION> %d %s', len(self.frame), body) return '' logging.debug('INVOCATION %d %s', len(self.frame), body) parts = splitParts(body) # title is the portion before the first | logging.debug('TITLE %s', parts[0].strip()) title = self.expandTemplates(parts[0].strip()) # SUBST # Apply the template tag to parameters without # substituting into them, e.g. # {{subst:t|a{{{p|q}}}b}} gives the wikitext start-a{{{p|q}}}b-end # @see https://www.mediawiki.org/wiki/Manual:Substitution#Partial_substitution subst = False if re.match(substWords, title, re.IGNORECASE): title = re.sub(substWords, '', title, 1, re.IGNORECASE) subst = True if title.lower() in self.magicWords.values: return self.magicWords[title.lower()] # Parser functions # The first argument is everything after the first colon. # It has been evaluated above. colon = title.find(':') if colon > 1: funct = title[:colon] parts[0] = title[colon+1:].strip() # side-effect (parts[0] not used later) # arguments after first are not evaluated ret = callParserFunction(funct, parts, self.frame) return self.expandTemplates(ret) title = fullyQualifiedTemplateTitle(title) if not title: self.template_title_errs += 1 return '' redirected = redirects.get(title) if redirected: title = redirected # get the template if title in templateCache: template = templateCache[title] elif title in templates: template = Template.parse(templates[title]) # add it to cache templateCache[title] = template del templates[title] else: # The page being included could not be identified return '' #logging.debug('TEMPLATE %s: %s', title, template) # tplarg = "{{{" parts "}}}" # parts = [ title *( "|" part ) ] # part = ( part-name "=" part-value ) / ( part-value ) # part-name = wikitext-L3 # part-value = wikitext-L3 # wikitext-L3 = literal / template / tplarg / link / comment / # line-eating-comment / unclosed-comment / # xmlish-element / *wikitext-L3 # A tplarg may contain other parameters as well as templates, e.g.: # {{{text|{{{quote|{{{1|{{error|Error: No text given}}}}}}}}}}} # hence no simple RE like this would work: # '{{{((?:(?!{{{).)*?)}}}' # We must use full CF parsing. # the parameter name itself might be computed, e.g.: # {{{appointe{{#if:{{{appointer14|}}}|r|d}}14|}}} # Because of the multiple uses of double-brace and triple-brace # syntax, expressions can sometimes be ambiguous. # Precedence rules specifed here: # http://www.mediawiki.org/wiki/Preprocessor_ABNF#Ideal_precedence # resolve ambiguities like this: # {{{{ }}}} -> { {{{ }}} } # {{{{{ }}}}} -> {{ {{{ }}} }} # # :see: https://en.wikipedia.org/wiki/Help:Template#Handling_parameters params = parts[1:] if not subst: # Evaluate parameters, since they may contain templates, including # the symbol "=". # {{#ifexpr: {{{1}}} = 1 }} params = [self.expandTemplates(p) for p in params] # build a dict of name-values for the parameter values params = self.templateParams(params) # Perform parameter substitution # extend frame before subst, since there may be recursion in default # parameter value, e.g. {{OTRS|celebrative|date=April 2015}} in article # 21637542 in enwiki. self.frame.append((title, params)) instantiated = template.subst(params, self) #logging.debug('instantiated %d %s', len(self.frame), instantiated) value = self.expandTemplates(instantiated) self.frame.pop() #logging.debug(' INVOCATION> %s %d %s', title, len(self.frame), value) return value # ---------------------------------------------------------------------- # parameter handling def splitParts(paramsList): """ :param paramList: the parts of a template or tplarg. Split template parameters at the separator "|". separator "=". Template parameters often contain URLs, internal links, text or even template expressions, since we evaluate templates outside in. This is required for cases like: {{#if: {{{1}}} | {{lc:{{{1}}} | "parameter missing"}} Parameters are separated by "|" symbols. However, we cannot simply split the string on "|" symbols, since these also appear inside templates and internal links, e.g. {{if:| |{{#if:the president| |{{#if:| [[Category:Hatnote templates|A{{PAGENAME}}]] }} }} }} We split parts at the "|" symbols that are not inside any pair {{{...}}}, {{...}}, [[...]], {|...|}. """ # Must consider '[' as normal in expansion of Template:EMedicine2: # #ifeq: ped|article|[http://emedicine.medscape.com/article/180-overview|[http://www.emedicine.com/ped/topic180.htm#{{#if: |section~}} # as part of: # {{#ifeq: ped|article|[http://emedicine.medscape.com/article/180-overview|[http://www.emedicine.com/ped/topic180.htm#{{#if: |section~}}}} ped/180{{#if: |~}}] # should handle both tpl arg like: # 4|{{{{{subst|}}}CURRENTYEAR}} # and tpl parameters like: # ||[[Category:People|{{#if:A|A|{{PAGENAME}}}}]] sep = '|' parameters = [] cur = 0 for s,e in findMatchingBraces(paramsList): par = paramsList[cur:s].split(sep) if par: if parameters: # portion before | belongs to previous parameter parameters[-1] += par[0] if len(par) > 1: # rest are new parameters parameters.extend(par[1:]) else: parameters = par elif not parameters: parameters = [''] # create first param # add span to last previous parameter parameters[-1] += paramsList[s:e] cur = e # leftover par = paramsList[cur:].split(sep) if par: if parameters: # portion before | belongs to previous parameter parameters[-1] += par[0] if len(par) > 1: # rest are new parameters parameters.extend(par[1:]) else: parameters = par #logging.debug('splitParts %s %s\nparams: %s', sep, paramsList, str(parameters)) return parameters def findMatchingBraces(text, ldelim=0): """ :param ldelim: number of braces to match. 0 means match [[]], {{}} and {{{}}}. """ # Parsing is done with respect to pairs of double braces {{..}} delimiting # a template, and pairs of triple braces {{{..}}} delimiting a tplarg. # If double opening braces are followed by triple closing braces or # conversely, this is taken as delimiting a template, with one left-over # brace outside it, taken as plain text. For any pattern of braces this # defines a set of templates and tplargs such that any two are either # separate or nested (not overlapping). # Unmatched double rectangular closing brackets can be in a template or # tplarg, but unmatched double rectangular opening brackets cannot. # Unmatched double or triple closing braces inside a pair of # double rectangular brackets are treated as plain text. # Other formulation: in ambiguity between template or tplarg on one hand, # and a link on the other hand, the structure with the rightmost opening # takes precedence, even if this is the opening of a link without any # closing, so not producing an actual link. # In the case of more than three opening braces the last three are assumed # to belong to a tplarg, unless there is no matching triple of closing # braces, in which case the last two opening braces are are assumed to # belong to a template. # We must skip individual { like in: # {{#ifeq: {{padleft:|1|}} | { | |  }} # We must resolve ambiguities like this: # {{{{ }}}} -> { {{{ }}} } # {{{{{ }}}}} -> {{ {{{ }}} }} # {{#if:{{{{{#if:{{{nominee|}}}|nominee|candidate}}|}}}|...}} # Handle: # {{{{{|safesubst:}}}#Invoke:String|replace|{{{1|{{{{{|safesubst:}}}PAGENAME}}}}}|%s+%([^%(]-%)$||plain=false}} # as well as expressions with stray }: # {{{link|{{ucfirst:{{{1}}}}}} interchange}}} if ldelim: # 2-3 reOpen = re.compile('[{]{%d,}' % ldelim) # at least ldelim reNext = re.compile('[{]{2,}|}{2,}') # at least 2 else: reOpen = re.compile('{{2,}|\[{2,}') reNext = re.compile('{{2,}|}{2,}|\[{2,}|]{2,}') # at least 2 cur = 0 while True: m1 = reOpen.search(text, cur) if not m1: return lmatch = m1.end()-m1.start() if m1.group()[0] == '{': stack = [lmatch] # stack of opening braces lengths else: stack = [-lmatch] # negative means [ end = m1.end() while True: m2 = reNext.search(text, end) if not m2: return # unbalanced end = m2.end() brac = m2.group()[0] lmatch = m2.end()-m2.start() if brac == '{': stack.append(lmatch) elif brac == '}': while stack: openCount = stack.pop() # opening span if openCount == 0: # illegal unmatched [[ continue if lmatch >= openCount: lmatch -= openCount if lmatch <= 1: # either close or stray } break else: # put back unmatched stack.append(openCount - lmatch) break if not stack: yield m1.start(), end-lmatch cur = end break elif len(stack) == 1 and 0 < stack[0] < ldelim: # ambiguous {{{{{ }}} }} yield m1.start() + stack[0], end cur = end break elif brac == '[': # [[ stack.append(-lmatch) else: # ]] while stack and stack[-1] < 0: # matching [[ openCount = -stack.pop() if lmatch >= openCount: lmatch -= openCount if lmatch <= 1: # either close or stray ] break else: # put back unmatched (negative) stack.append(lmatch - openCount) break if not stack: yield m1.start(), end-lmatch cur = end break # unmatched ]] are discarded cur = end def findBalanced(text, openDelim, closeDelim): """ Assuming that text contains a properly balanced expression using :param openDelim: as opening delimiters and :param closeDelim: as closing delimiters. :return: an iterator producing pairs (start, end) of start and end positions in text containing a balanced expression. """ openPat = '|'.join([re.escape(x) for x in openDelim]) # patter for delimiters expected after each opening delimiter afterPat = { o:re.compile(openPat+'|'+c, re.DOTALL) for o,c in izip(openDelim, closeDelim)} stack = [] start = 0 cur = 0 end = len(text) startSet = False startPat = re.compile(openPat) nextPat = startPat while True: next = nextPat.search(text, cur) if not next: return if not startSet: start = next.start() startSet = True delim = next.group(0) if delim in openDelim: stack.append(delim) nextPat = afterPat[delim] else: opening = stack.pop() # assert opening == openDelim[closeDelim.index(next.group(0))] if stack: nextPat = afterPat[stack[-1]] else: yield start, next.end() nextPat = startPat start = next.end() startSet = False cur = next.end() # ---------------------------------------------------------------------- # Modules # Only minimal support # FIXME: import Lua modules. modules = { 'convert' : { 'convert': lambda x, u, *rest: x+' '+u, # no conversion } } # ---------------------------------------------------------------------- # variables class MagicWords(object): """ One copy in each Extractor. @see https://doc.wikimedia.org/mediawiki-core/master/php/MagicWord_8php_source.html """ names = [ '!', 'currentmonth', 'currentmonth1', 'currentmonthname', 'currentmonthnamegen', 'currentmonthabbrev', 'currentday', 'currentday2', 'currentdayname', 'currentyear', 'currenttime', 'currenthour', 'localmonth', 'localmonth1', 'localmonthname', 'localmonthnamegen', 'localmonthabbrev', 'localday', 'localday2', 'localdayname', 'localyear', 'localtime', 'localhour', 'numberofarticles', 'numberoffiles', 'numberofedits', 'articlepath', 'pageid', 'sitename', 'server', 'servername', 'scriptpath', 'stylepath', 'pagename', 'pagenamee', 'fullpagename', 'fullpagenamee', 'namespace', 'namespacee', 'namespacenumber', 'currentweek', 'currentdow', 'localweek', 'localdow', 'revisionid', 'revisionday', 'revisionday2', 'revisionmonth', 'revisionmonth1', 'revisionyear', 'revisiontimestamp', 'revisionuser', 'revisionsize', 'subpagename', 'subpagenamee', 'talkspace', 'talkspacee', 'subjectspace', 'subjectspacee', 'talkpagename', 'talkpagenamee', 'subjectpagename', 'subjectpagenamee', 'numberofusers', 'numberofactiveusers', 'numberofpages', 'currentversion', 'rootpagename', 'rootpagenamee', 'basepagename', 'basepagenamee', 'currenttimestamp', 'localtimestamp', 'directionmark', 'contentlanguage', 'numberofadmins', 'cascadingsources', ] def __init__(self): self.values = {} self.values['!'] = '|' def __getitem__(self, name): return self.values.get(name) def __setitem__(self, name, value): self.values[name] = value switches = [ '__NOTOC__', '__FORCETOC__', '__TOC__', '__TOC__', '__NEWSECTIONLINK__', '__NONEWSECTIONLINK__', '__NOGALLERY__', '__HIDDENCAT__', '__NOCONTENTCONVERT__', '__NOCC__', '__NOTITLECONVERT__', '__NOTC__', '__START__', '__END__', '__INDEX__', '__NOINDEX__', '__STATICREDIRECT__', '__DISAMBIG__' ] magicWordsRE = re.compile('|'.join(MagicWords.switches)) # ---------------------------------------------------------------------- # parser functions utilities def ucfirst(string): """:return: a string with just its first character uppercase We can't use title() since it coverts all words. """ if string: if len(string) > 1: return string[0].upper() + string[1:] else: return string.upper() else: return '' def lcfirst(string): """:return: a string with its first character lowercase""" if string: if len(string) > 1: return string[0].lower() + string[1:] else: return string.lower() else: return '' def fullyQualifiedTemplateTitle(templateTitle): """ Determine the namespace of the page being included through the template mechanism """ global templatePrefix if templateTitle.startswith(':'): # Leading colon by itself implies main namespace, so strip this colon return ucfirst(templateTitle[1:]) else: m = re.match('([^:]*)(:.*)', templateTitle) if m: # colon found but not in the first position - check if it # designates a known namespace prefix = normalizeNamespace(m.group(1)) if prefix in knownNamespaces: return prefix + ucfirst(m.group(2)) # The title of the page being included is NOT in the main namespace and # lacks any other explicit designation of the namespace - therefore, it # is resolved to the Template namespace (that's the default for the # template inclusion mechanism). # This is a defense against pages whose title only contains UTF-8 chars # that are reduced to an empty string. Right now I can think of one such # case - which represents the non-breaking space. # In this particular case, this page is a redirect to [[Non-nreaking # space]], but having in the system a redirect page with an empty title # causes numerous problems, so we'll live happier without it. if templateTitle: return templatePrefix + ucfirst(templateTitle) else: return '' # caller may log as error def normalizeNamespace(ns): return ucfirst(ns) # ---------------------------------------------------------------------- # Parser functions # see http://www.mediawiki.org/wiki/Help:Extension:ParserFunctions # https://github.com/Wikia/app/blob/dev/extensions/ParserFunctions/ParserFunctions_body.php class Infix: """Infix operators. The calling sequence for the infix is: x |op| y """ def __init__(self, function): self.function = function def __ror__(self, other): return Infix(lambda x, self=self, other=other: self.function(other, x)) def __or__(self, other): return self.function(other) def __rlshift__(self, other): return Infix(lambda x, self=self, other=other: self.function(other, x)) def __rshift__(self, other): return self.function(other) def __call__(self, value1, value2): return self.function(value1, value2) ROUND = Infix(lambda x,y: round(x, y)) def sharp_expr(expr): try: expr = re.sub('=', '==', expr) expr = re.sub('mod', '%', expr) expr = re.sub('\bdiv\b', '/', expr) expr = re.sub('\bround\b', '|ROUND|', expr) return unicode(eval(expr)) except: return '' def sharp_if(testValue, valueIfTrue, valueIfFalse=None, *args): # In theory, we should evaluate the first argument here, # but it was evaluated while evaluating part[0] in expandTemplate(). if testValue.strip(): # The {{#if:}} function is an if-then-else construct. # The applied condition is: "The condition string is non-empty". valueIfTrue = valueIfTrue.strip() if valueIfTrue: return valueIfTrue elif valueIfFalse: return valueIfFalse.strip() return "" def sharp_ifeq(lvalue, rvalue, valueIfTrue, valueIfFalse=None, *args): rvalue = rvalue.strip() if rvalue: # lvalue is always defined if lvalue.strip() == rvalue: # The {{#ifeq:}} function is an if-then-else construct. The # applied condition is "is rvalue equal to lvalue". Note that this # does only string comparison while MediaWiki implementation also # supports numerical comparissons. if valueIfTrue: return valueIfTrue.strip() else: if valueIfFalse: return valueIfFalse.strip() return "" def sharp_iferror(test, then='', Else=None, *args): if re.match('<(?:strong|span|p|div)\s(?:[^\s>]*\s+)*?class="(?:[^"\s>]*\s+)*?error(?:\s[^">]*)?"', test): return then elif Else is None: return test.strip() else: return Else.strip() def sharp_switch(primary, *params): # FIXME: we don't support numeric expressions in primary # {{#switch: comparison string # | case1 = result1 # | case2 # | case4 = result2 # | 1 | case5 = result3 # | #default = result4 # }} primary = primary.strip() found = False # for fall through cases default = None rvalue = None lvalue = '' for param in params: # handle cases like: # #default = [http://www.perseus.tufts.edu/hopper/text?doc=Perseus...] pair = param.split('=', 1) lvalue = pair[0].strip() rvalue = None if len(pair) > 1: # got "=" rvalue = pair[1].strip() # check for any of multiple values pipe separated if found or primary in [v.strip() for v in lvalue.split('|')]: # Found a match, return now return rvalue elif lvalue == '#default': default = rvalue rvalue = None # avoid defaulting to last case elif lvalue == primary: # If the value matches, set a flag and continue found = True # Default case # Check if the last item had no = sign, thus specifying the default case if rvalue is not None: return lvalue elif default is not None: return default return '' # Extension Scribuntu def sharp_invoke(module, function, frame): functions = modules.get(module) if functions: funct = functions.get(function) if funct: # find parameters in frame whose title is the one of the original # template invocation templateTitle = fullyQualifiedTemplateTitle(function) if not templateTitle: logging.warn("Template with empty title") pair = next((x for x in frame if x[0] == templateTitle), None) if pair: params = pair[1] # extract positional args params = [params.get(str(i+1)) for i in range(len(params))] return funct(*params) else: return funct() return '' parserFunctions = { '#expr': sharp_expr, '#if': sharp_if, '#ifeq': sharp_ifeq, '#iferror': sharp_iferror, '#ifexpr': lambda *args: '', # not supported '#ifexist': lambda *args: '', # not supported '#rel2abs': lambda *args: '', # not supported '#switch': sharp_switch, '#language': lambda *args: '', # not supported '#time': lambda *args: '', # not supported '#timel': lambda *args: '', # not supported '#titleparts': lambda *args: '', # not supported # This function is used in some pages to construct links # http://meta.wikimedia.org/wiki/Help:URL 'urlencode': lambda string, *rest: urllib.quote(string.encode('utf-8')), 'lc': lambda string, *rest: string.lower() if string else '', 'lcfirst': lambda string, *rest: lcfirst(string), 'uc': lambda string, *rest: string.upper() if string else '', 'ucfirst': lambda string, *rest: ucfirst(string), 'int': lambda string, *rest: str(int(string)), } def callParserFunction(functionName, args, frame): """ Parser functions have similar syntax as templates, except that the first argument is everything after the first colon. :return: the result of the invocation, None in case of failure. http://meta.wikimedia.org/wiki/Help:ParserFunctions """ try: if functionName == '#invoke': # special handling of frame ret = sharp_invoke(args[0].strip(), args[1].strip(), frame) #logging.debug('parserFunction> %s %s', functionName, ret) return ret if functionName in parserFunctions: ret = parserFunctions[functionName](*args) #logging.debug('parserFunction> %s %s', functionName, ret) return ret except: return "" # FIXME: fix errors return "" # ---------------------------------------------------------------------- # Expand using WikiMedia API # import json # def expandTemplates(text): # """Expand templates invoking MediaWiki API""" # text = urlib.urlencodew(text.encode('utf-8')) # base = urlbase[:urlbase.rfind('/')] # url = base + "/w/api.php?action=expandtemplates&format=json&text=" + text # exp = json.loads(urllib.urlopen(url)) # return exp['expandtemplates']['*'] # ---------------------------------------------------------------------- # Extract Template definition reNoinclude = re.compile(r'(?:.*?)', re.DOTALL) reIncludeonly = re.compile(r'|', re.DOTALL) # These are built before spawning processes, hence thay are shared. templates = {} redirects = {} # cache of parser templates # FIXME: sharing this with a Manager slows down. templateCache = {} def define_template(title, page): """ Adds a template defined in the :param page:. @see https://en.wikipedia.org/wiki/Help:Template#Noinclude.2C_includeonly.2C_and_onlyinclude """ global templates global redirects #title = normalizeTitle(title) # check for redirects m = re.match('#REDIRECT.*?\[\[([^\]]*)]]', page[0], re.IGNORECASE) if m: redirects[title] = m.group(1) #normalizeTitle(m.group(1)) return text = unescape(''.join(page)) # We're storing template text for future inclusion, therefore, # remove all text and keep all text # (but eliminate tags per se). # However, if ... parts are present, # then only keep them and discard the rest of the template body. # This is because using on a text fragment is # equivalent to enclosing it in tags **AND** # enclosing all the rest of the template body in tags. # remove comments text = comment.sub('', text) # eliminate fragments text = reNoinclude.sub('', text) # eliminate unterminated elements text = re.sub(r'.*$', '', text, flags=re.DOTALL) text = re.sub(r'', '', text) onlyincludeAccumulator = '' for m in re.finditer('(.*?)', text, re.DOTALL): onlyincludeAccumulator += m.group(1) if onlyincludeAccumulator: text = onlyincludeAccumulator else: text = reIncludeonly.sub('', text) if text: if title in templates: logging.warn('Redefining: %s', title) templates[title] = text # ---------------------------------------------------------------------- def dropNested(text, openDelim, closeDelim): """ A matching function for nested expressions, e.g. namespaces and tables. """ openRE = re.compile(openDelim, re.IGNORECASE) closeRE = re.compile(closeDelim, re.IGNORECASE) # partition text in separate blocks { } { } spans = [] # pairs (s, e) for each partition nest = 0 # nesting level start = openRE.search(text, 0) if not start: return text end = closeRE.search(text, start.end()) next = start while end: next = openRE.search(text, next.end()) if not next: # termination while nest: # close all pending nest -=1 end0 = closeRE.search(text, end.end()) if end0: end = end0 else: break spans.append((start.start(), end.end())) break while end.end() < next.start(): # { } { if nest: nest -= 1 # try closing more last = end.end() end = closeRE.search(text, end.end()) if not end: # unbalanced if spans: span = (spans[0][0], last) else: span = (start.start(), last) spans = [span] break else: spans.append((start.start(), end.end())) # advance start, find next close start = next end = closeRE.search(text, next.end()) break # { } if next != start: # { { } nest += 1 # collect text outside partitions return dropSpans(spans, text) def dropSpans(spans, text): """ Drop from text the blocks identified in :param spans:, possibly nested. """ spans.sort() res = '' offset = 0 for s, e in spans: if offset <= s: # handle nesting if offset < s: res += text[offset:s] offset = e res += text[offset:] return res # ---------------------------------------------------------------------- # WikiLinks # See https://www.mediawiki.org/wiki/Help:Links#Internal_links # Can be nested [[File:..|..[[..]]..|..]], [[Category:...]], etc. # Also: [[Help:IPA for Catalan|[andora]]] def replaceInternalLinks(text): """ Replaces external links of the form: [[title |...|label]]trail with title concatenated with trail, when present, e.g. 's' for plural. """ # call this after removal of external links, so we need not worry about # triple closing ]]]. cur = 0 res = '' for s,e in findBalanced(text, ['[['], [']]']): m = tailRE.match(text, e) if m: trail = m.group(0) end = m.end() else: trail = '' end = e inner = text[s+2:e-2] # find first | pipe = inner.find('|') if pipe < 0: title = inner label = title else: title = inner[:pipe].rstrip() # find last | curp = pipe+1 for s1,e1 in findBalanced(inner, ['[['], [']]']): last = inner.rfind('|', curp, s1) if last >= 0: pipe = last # advance curp = e1 label = inner[pipe+1:].strip() res += text[cur:s] + makeInternalLink(title, label) + trail cur = end return res + text[cur:] # the official version is a method in class Parser, similar to this: # def replaceInternalLinks2(text): # global wgExtraInterlanguageLinkPrefixes # # the % is needed to support urlencoded titles as well # tc = Title::legalChars() + '#%' # # Match a link having the form [[namespace:link|alternate]]trail # e1 = re.compile("([%s]+)(?:\\|(.+?))?]](.*)" % tc, re.S | re.D) # # Match cases where there is no "]]", which might still be images # e1_img = re.compile("([%s]+)\\|(.*)" % tc, re.S | re.D) # holders = LinkHolderArray(self) # # split the entire text string on occurrences of [[ # iterBrackets = re.compile('[[').finditer(text) # m in iterBrackets.next() # # get the first element (all text up to first [[) # s = text[:m.start()] # cur = m.end() # line = s # useLinkPrefixExtension = self.getTargetLanguage().linkPrefixExtension() # e2 = None # if useLinkPrefixExtension: # # Match the end of a line for a word that is not followed by whitespace, # # e.g. in the case of "The Arab al[[Razi]]", "al" will be matched # global wgContLang # charset = wgContLang.linkPrefixCharset() # e2 = re.compile("((?>.*[^charset]|))(.+)", re.S | re.D | re.U) # if self.mTitle is None: # raise MWException(__METHOD__ + ": \self.mTitle is null\n") # nottalk = not self.mTitle.isTalkPage() # if useLinkPrefixExtension: # m = e2.match(s) # if m: # first_prefix = m.group(2) # else: # first_prefix = false # else: # prefix = '' # useSubpages = self.areSubpagesAllowed() # for m in iterBrackets: # line = text[cur:m.start()] # cur = m.end() # # TODO: Check for excessive memory usage # if useLinkPrefixExtension: # m = e2.match(e2) # if m: # prefix = m.group(2) # s = m.group(1) # else: # prefix = '' # # first link # if first_prefix: # prefix = first_prefix # first_prefix = False # might_be_img = False # m = e1.match(line) # if m: # page with normal label or alt # label = m.group(2) # # If we get a ] at the beginning of m.group(3) that means we have a link that is something like: # # [[Image:Foo.jpg|[http://example.com desc]]] <- having three ] in a row fucks up, # # the real problem is with the e1 regex # # See bug 1300. # # # # Still some problems for cases where the ] is meant to be outside punctuation, # # and no image is in sight. See bug 2095. # # # if label and m.group(3)[0] == ']' and '[' in label: # label += ']' # so that replaceExternalLinks(label) works later # m.group(3) = m.group(3)[1:] # # fix up urlencoded title texts # if '%' in m.group(1): # # Should anchors '#' also be rejected? # m.group(1) = str_replace(array('<', '>'), array('<', '>'), rawurldecode(m.group(1))) # trail = m.group(3) # else: # m = e1_img.match(line): # if m: # # Invalid, but might be an image with a link in its caption # might_be_img = true # label = m.group(2) # if '%' in m.group(1): # m.group(1) = rawurldecode(m.group(1)) # trail = "" # else: # Invalid form; output directly # s += prefix + '[[' + line # continue # origLink = m.group(1) # # Dont allow internal links to pages containing # # PROTO: where PROTO is a valid URL protocol these # # should be external links. # if (preg_match('/^(?i:' + self.mUrlProtocols + ')/', origLink)) { # s += prefix + '[[' + line # continue # } # # Make subpage if necessary # if useSubpages: # link = self.maybeDoSubpageLink(origLink, label) # else: # link = origLink # noforce = origLink[0] != ':' # if not noforce: # # Strip off leading ':' # link = link[1:] # nt = Title::newFromText(self.mStripState.unstripNoWiki(link)) # if nt is None: # s += prefix + '[[' + line # continue # ns = nt.getNamespace() # iw = nt.getInterwiki() # if might_be_img { # if this is actually an invalid link # if (ns == NS_FILE and noforce) { # but might be an image # found = False # while True: # # look at the next 'line' to see if we can close it there # next_line = iterBrakets.next() # if not next_line: # break # m = explode(']]', next_line, 3) # if m.lastindex == 3: # # the first ]] closes the inner link, the second the image # found = True # label += "[[%s]]%s" % (m.group(0), m.group(1)) # trail = m.group(2) # break # elif m.lastindex == 2: # # if there is exactly one ]] that is fine, we will keep looking # label += "[[{m[0]}]]{m.group(1)}" # else: # # if next_line is invalid too, we need look no further # label += '[[' + next_line # break # if not found: # # we couldnt find the end of this imageLink, so output it raw # # but dont ignore what might be perfectly normal links in the text we ve examined # holders.merge(self.replaceInternalLinks2(label)) # s += "{prefix}[[%s|%s" % (link, text) # # note: no trail, because without an end, there *is* no trail # continue # } else: # it is not an image, so output it raw # s += "{prefix}[[%s|%s" % (link, text) # # note: no trail, because without an end, there *is* no trail # continue # } # wasblank = (text == '') # if wasblank: # text = link # else: # # Bug 4598 madness. Handle the quotes only if they come from the alternate part # # [[Lista d''e paise d''o munno]] . Lista d''e paise d''o munno # # [[Criticism of Harry Potter|Criticism of ''Harry Potter'']] # # . Criticism of Harry Potter # text = self.doQuotes(text) # # Link not escaped by : , create the various objects # if noforce and not nt.wasLocalInterwiki(): # # Interwikis # if iw and mOptions.getInterwikiMagic() and nottalk and ( # Language::fetchLanguageName(iw, None, 'mw') or # in_array(iw, wgExtraInterlanguageLinkPrefixes)): # # Bug 24502: filter duplicates # if iw not in mLangLinkLanguages: # self.mLangLinkLanguages[iw] = True # self.mOutput.addLanguageLink(nt.getFullText()) # s = rstrip(s + prefix) # s += strip(trail, "\n") == '' ? '': prefix + trail # continue # if ns == NS_FILE: # if not wfIsBadImage(nt.getDBkey(), self.mTitle): # if wasblank: # # if no parameters were passed, text # # becomes something like "File:Foo.png", # # which we dont want to pass on to the # # image generator # text = '' # else: # # recursively parse links inside the image caption # # actually, this will parse them in any other parameters, too, # # but it might be hard to fix that, and it doesnt matter ATM # text = self.replaceExternalLinks(text) # holders.merge(self.replaceInternalLinks2(text)) # # cloak any absolute URLs inside the image markup, so replaceExternalLinks() wont touch them # s += prefix + self.armorLinks( # self.makeImage(nt, text, holders)) + trail # else: # s += prefix + trail # continue # if ns == NS_CATEGORY: # s = rstrip(s + "\n") # bug 87 # if wasblank: # sortkey = self.getDefaultSort() # else: # sortkey = text # sortkey = Sanitizer::decodeCharReferences(sortkey) # sortkey = str_replace("\n", '', sortkey) # sortkey = self.getConverterLanguage().convertCategoryKey(sortkey) # self.mOutput.addCategory(nt.getDBkey(), sortkey) # s += strip(prefix + trail, "\n") == '' ? '' : prefix + trail # continue # } # } # # Self-link checking. For some languages, variants of the title are checked in # # LinkHolderArray::doVariants() to allow batching the existence checks necessary # # for linking to a different variant. # if ns != NS_SPECIAL and nt.equals(self.mTitle) and !nt.hasFragment(): # s += prefix + Linker::makeSelfLinkObj(nt, text, '', trail) # continue # # NS_MEDIA is a pseudo-namespace for linking directly to a file # # @todo FIXME: Should do batch file existence checks, see comment below # if ns == NS_MEDIA: # # Give extensions a chance to select the file revision for us # options = [] # descQuery = False # Hooks::run('BeforeParserFetchFileAndTitle', # [this, nt, &options, &descQuery]) # # Fetch and register the file (file title may be different via hooks) # file, nt = self.fetchFileAndTitle(nt, options) # # Cloak with NOPARSE to avoid replacement in replaceExternalLinks # s += prefix + self.armorLinks( # Linker::makeMediaLinkFile(nt, file, text)) + trail # continue # # Some titles, such as valid special pages or files in foreign repos, should # # be shown as bluelinks even though they are not included in the page table # # # # @todo FIXME: isAlwaysKnown() can be expensive for file links; we should really do # # batch file existence checks for NS_FILE and NS_MEDIA # if iw == '' and nt.isAlwaysKnown(): # self.mOutput.addLink(nt) # s += self.makeKnownLinkHolder(nt, text, array(), trail, prefix) # else: # # Links will be added to the output link list after checking # s += holders.makeHolder(nt, text, array(), trail, prefix) # } # return holders def makeInternalLink(title, label): colon = title.find(':') if colon > 0 and title[:colon] not in acceptedNamespaces: return '' if colon == 0: # drop also :File: colon2 = title.find(':', colon+1) if colon2 > 1 and title[colon+1:colon2] not in acceptedNamespaces: return '' if Extractor.keepLinks: return '%s' % (urllib.quote(title.encode('utf-8')), label) else: return label # ---------------------------------------------------------------------- # External links # from: https://doc.wikimedia.org/mediawiki-core/master/php/DefaultSettings_8php_source.html wgUrlProtocols = [ 'bitcoin:', 'ftp://', 'ftps://', 'geo:', 'git://', 'gopher://', 'http://', 'https://', 'irc://', 'ircs://', 'magnet:', 'mailto:', 'mms://', 'news:', 'nntp://', 'redis://', 'sftp://', 'sip:', 'sips:', 'sms:', 'ssh://', 'svn://', 'tel:', 'telnet://', 'urn:', 'worldwind://', 'xmpp:', '//' ] # from: https://doc.wikimedia.org/mediawiki-core/master/php/Parser_8php_source.html # Constants needed for external link processing # Everything except bracket, space, or control characters # \p{Zs} is unicode 'separator, space' category. It covers the space 0x20 # as well as U+3000 is IDEOGRAPHIC SPACE for bug 19052 EXT_LINK_URL_CLASS = r'[^][<>"\x00-\x20\x7F\s]' ExtLinkBracketedRegex = re.compile('\[(((?i)' + '|'.join(wgUrlProtocols) + ')' + EXT_LINK_URL_CLASS + r'+)\s*([^\]\x00-\x08\x0a-\x1F]*?)\]', re.S | re.U) EXT_IMAGE_REGEX = re.compile( r"""^(http://|https://)([^][<>"\x00-\x20\x7F\s]+) /([A-Za-z0-9_.,~%\-+&;#*?!=()@\x80-\xFF]+)\.((?i)gif|png|jpg|jpeg)$""", re.X | re.S | re.U) def replaceExternalLinks(text): s = '' cur = 0 for m in ExtLinkBracketedRegex.finditer(text): s += text[cur:m.start()] cur = m.end() url = m.group(1) label = m.group(3) # # The characters '<' and '>' (which were escaped by # # removeHTMLtags()) should not be included in # # URLs, per RFC 2396. # m2 = re.search('&(lt|gt);', url) # if m2: # link = url[m2.end():] + ' ' + link # url = url[0:m2.end()] # If the link text is an image URL, replace it with an tag # This happened by accident in the original parser, but some people used it extensively m = EXT_IMAGE_REGEX.match(label) if m: label = makeExternalImage(label) # Use the encoded URL # This means that users can paste URLs directly into the text # Funny characters like ö aren't valid in URLs anyway # This was changed in August 2004 s += makeExternalLink(url, label) #+ trail return s + text[cur:] # Function applied to wikiLinks def makeExternalLink(title, anchor): colon = title.find(':') if colon > 0 and title[:colon] not in acceptedNamespaces: return '' if colon == 0: # drop also :File: colon2 = title.find(':', colon+1) if colon2 > 1 and title[colon+1:colon2] not in acceptedNamespaces: return '' if Extractor.keepLinks: return '%s' % (urllib.quote(title.encode('utf-8')), anchor) else: return anchor def makeExternalImage(url, alt=''): if Extractor.keepLinks: return '%s' % (url, alt) else: return alt # ---------------------------------------------------------------------- # match tail after wikilink tailRE = re.compile('\w+') syntaxhighlight = re.compile('<syntaxhighlight .*?>(.*?)</syntaxhighlight>', re.DOTALL) expand_templates = True def clean(extractor, text): """ Transforms wiki markup. @see https://www.mediawiki.org/wiki/Help:Formatting """ if (expand_templates): # expand templates # See: http://www.mediawiki.org/wiki/Help:Templates text = extractor.expandTemplates(text) else: # Drop transclusions (template, parser functions) text = dropNested(text, r'{{', r'}}') # Drop tables text = dropNested(text, r'{\|', r'\|}') # replace external links text = replaceExternalLinks(text) # replace internal links text = replaceInternalLinks(text) # drop MagicWords behavioral switches text = magicWordsRE.sub('', text) ################ Process HTML ############### # turn into HTML, except for the content of res = '' cur = 0 for m in syntaxhighlight.finditer(text): end = m.end() res += unescape(text[cur:m.start()]) + m.group(1) cur = end text = res + unescape(text[cur:]) # Handle bold/italic/quote if extractor.toHTML: text = bold_italic.sub(r'\1', text) text = bold.sub(r'\1', text) text = italic.sub(r'\1', text) else: text = bold_italic.sub(r'\1', text) text = bold.sub(r'\1', text) text = italic_quote.sub(r'"\1"', text) text = italic.sub(r'"\1"', text) text = quote_quote.sub(r'"\1"', text) # residuals of unbalanced quotes text = text.replace("'''", '').replace("''", '"') # Collect spans spans = [] # Drop HTML comments for m in comment.finditer(text): spans.append((m.start(), m.end())) # Drop self-closing tags for pattern in selfClosing_tag_patterns: for m in pattern.finditer(text): spans.append((m.start(), m.end())) # Drop ignored tags for left, right in ignored_tag_patterns: for m in left.finditer(text): spans.append((m.start(), m.end())) for m in right.finditer(text): spans.append((m.start(), m.end())) # Bulk remove all spans text = dropSpans(spans, text) # Drop discarded elements for tag in discardElements: text = dropNested(text, r'<\s*%s\b[^>/]*>' % tag, r'<\s*/\s*%s>' % tag) if not extractor.toHTML: # Turn into text what is left (&nbsp;) and text = unescape(text) # Expand placeholders for pattern, placeholder in placeholder_tag_patterns: index = 1 for match in pattern.finditer(text): text = text.replace(match.group(), '%s_%d' % (placeholder, index)) index += 1 text = text.replace('<<', u'«').replace('>>', u'»') ############################################# # Cleanup text text = text.replace('\t', ' ') text = spaces.sub(' ', text) text = dots.sub('...', text) text = re.sub(u' (,:\.\)\]»)', r'\1', text) text = re.sub(u'(\[\(«) ', r'\1', text) text = re.sub(r'\n\W+?\n', '\n', text, flags=re.U) # lines with only punctuations text = text.replace(',,', ',').replace(',.', '.') return text # skip level 1, it is page name level section = re.compile(r'(==+)\s*(.*?)\s*\1') listOpen = { '*': '

    ', '#': '
      ', ';': '
      ', ':': '
      ' } listClose = { '*': '
', '#': '', ';': '', ':': '' } listItem = { '*': '
  • %s
    • ', '#': '
  • %s', ';': '
    %s
    ', ':': '
    %s
    ' } def compact(text): """Deal with headers, lists, empty sections, residuals of tables. :param toHTML: convert to HTML """ page = [] # list of paragraph headers = {} # Headers for unfilled sections emptySection = False # empty sections are discarded listLevel = '' # nesting of lists for line in text.split('\n'): if not line: continue # Handle section titles m = section.match(line) if m: title = m.group(2) lev = len(m.group(1)) if Extractor.toHTML: page.append("%s" % (lev, title, lev)) if title and title[-1] not in '!?': title += '.' headers[lev] = title # drop previous headers for i in headers.keys(): if i > lev: del headers[i] emptySection = True continue # Handle page title if line.startswith('++'): title = line[2:-2] if title: if title[-1] not in '!?': title += '.' page.append(title) # handle indents elif line[0] == ':': #page.append(line.lstrip(':*#;')) continue # handle lists elif line[0] in '*#;:': if Extractor.toHTML: i = 0 for c,n in izip_longest(listLevel, line, fillvalue=''): if not n or n not in '*#;:': if c: page.append(listClose[c]) listLevel = listLevel[:-1] continue else: break # n != '' if c != n and (not c or (c not in ';:' and n not in ';:')): if c: # close level page.append(listClose[c]) listLevel = listLevel[:-1] listLevel += n page.append(listOpen[n]) i += 1 n = line[i-1] # last list char line = line[i:].strip() if line: # FIXME: n is '"' page.append(listItem[n] % line) else: continue elif len(listLevel): for c in reversed(listLevel): page.append(listClose[c]) listLevel = [] # Drop residuals of lists elif line[0] in '{|' or line[-1] == '}': continue # Drop irrelevant lines elif (line[0] == '(' and line[-1] == ')') or line.strip('.-') == '': continue elif len(headers): if not Extractor.keepSections: items = headers.items() items.sort() for (i, v) in items: page.append(v) headers.clear() page.append(line) # first line emptySection = False elif not emptySection: page.append(line) # dangerous # # Drop preformatted # elif line[0] == ' ': # continue return page def handle_unicode(entity): numeric_code = int(entity[2:-1]) if numeric_code >= 0x10000: return '' return unichr(numeric_code) #------------------------------------------------------------------------------ # Output class NextFile(object): """ Synchronous generation of next available file name. """ filesPerDir = 100 def __init__(self, path_name): self.path_name = path_name self.dir_index = -1 self.file_index = -1 def next(self): self.file_index = (self.file_index + 1) % NextFile.filesPerDir if self.file_index == 0: self.dir_index += 1 dirname = self._dirname() if not os.path.isdir(dirname): os.makedirs(dirname) return self._filepath() def _dirname(self): char1 = self.dir_index % 26 char2 = self.dir_index / 26 % 26 return os.path.join(self.path_name, '%c%c' % (ord('A') + char2, ord('A') + char1)) def _filepath(self): return '%s/wiki_%02d' % (self._dirname(), self.file_index) class OutputSplitter(object): """ File-like object, that splits output to multiple files of a given max size. """ def __init__(self, nextFile, max_file_size=0, compress=True): """ :param nextfile: a NextFile object from which to obtain filenames to use. :param max_file_size: the maximum size of each file. :para compress: whether to write data with bzip compression. """ self.nextFile = nextFile self.compress = compress self.max_file_size = max_file_size self.file = self.open(self.nextFile.next()) def reserve(self, size): if self.file.tell() + size > self.max_file_size: self.close() self.file = self.open(self.nextFile.next()) def write(self, data): self.reserve(len(data)) self.file.write(data) def close(self): self.file.close() def open(self, filename): if self.compress: return bz2.BZ2File(filename + '.bz2', 'w') else: return open(filename, 'w') # ---------------------------------------------------------------------- # READER tagRE = re.compile(r'(.*?)<(/?\w+)[^>]*>(?:([^<]*)(<.*?>)?)?') # 1 2 3 4 def load_templates(file, output_file=None): """ Load templates from :param file:. :param output_file: file where to save templates and modules. """ global templateNamespace, templatePrefix templatePrefix = templateNamespace + ':' global moduleNamespace, modulePrefix modulePrefix = moduleNamespace + ':' articles = 0 page = [] inText = False if output_file: output = codecs.open(output_file, 'wb', 'utf-8') for line in file: line = line.decode('utf-8') if '<' not in line: # faster than doing re.search() if inText: page.append(line) continue m = tagRE.search(line) if not m: continue tag = m.group(2) if tag == 'page': page = [] elif tag == 'title': title = m.group(3) elif tag == 'text': inText = True line = line[m.start(3):m.end(3)] page.append(line) if m.lastindex == 4: # open-close inText = False elif tag == '/text': if m.group(1): page.append(m.group(1)) inText = False elif inText: page.append(line) elif tag == '/page': if not output_file and not templateNamespace: # do not know it yet # we reconstruct it from the first title colon = title.find(':') if colon > 1: templateNamespace = title[:colon] templatePrefix = title[:colon+1] # FIXME: should reconstruct also moduleNamespace if title.startswith(templatePrefix): define_template(title, page) # save templates and modules to file if output_file and (title.startswith(templatePrefix) or title.startswith(modulePrefix)): output.write('\n') output.write(' %s\n' % title) output.write(' 10\n') output.write(' ') for line in page: output.write(line) output.write(' \n') output.write('\n') page = [] articles += 1 if articles % 100000 == 0: logging.info("Preprocessed %d pages", articles) if output_file: output.close() logging.info("Saved %d templates to '%s'", len(templates), output_file) def process_dump(input_file, template_file, out_file, file_size, file_compress, process_count): """ :param input_file: name of the wikipedia dump file; '-' to read from stdin :param template_file: optional file with template definitions. :param out_file: directory where to store extracted data, or '-' for stdout :param file_size: max size of each extracted file, or None for no max (one file) :param file_compress: whether to compress files with bzip. :param process_count: number of extraction processes to spawn. """ global urlbase global knownNamespaces global templateNamespace, templatePrefix global moduleNamespace, modulePrefix global expand_templates if input_file == '-': input = sys.stdin else: input = fileinput.FileInput(input_file, openhook=fileinput.hook_compressed) # collect siteinfo for line in input: line = line.decode('utf-8') m = tagRE.search(line) if not m: continue tag = m.group(2) if tag == 'base': # discover urlbase from the xml dump file # /mediawiki/siteinfo/base base = m.group(3) urlbase = base[:base.rfind("/")] elif tag == 'namespace': knownNamespaces.add(m.group(3)) if re.search('key="10"', line): templateNamespace = m.group(3) templatePrefix = templateNamespace + ':' elif re.search('key="828"', line): moduleNamespace = m.group(3) modulePrefix = moduleNamespace + ':' elif tag == '/siteinfo': break if expand_templates: # preprocess template_load_start = default_timer() if template_file and os.path.exists(template_file): logging.info("Preprocessing '%s' to collect template definitions: this may take some time.", template_file) file = fileinput.FileInput(template_file, openhook=fileinput.hook_compressed) load_templates(file) file.close() else: if input_file == '-': # can't scan then reset stdin; must error w/ suggestion to specify template_file raise ValueError("to use templates with stdin dump, must supply explicit template-file") logging.info("Preprocessing '%s' to collect template definitions: this may take some time.", input_file) load_templates(input, template_file) input.close() input = fileinput.FileInput(input_file,openhook=fileinput.hook_compressed) template_load_elapsed = default_timer() - template_load_start logging.info("Loaded %d templates in %.1fs", len(templates), template_load_elapsed) if out_file == '-': output = sys.stdout if file_compress: logging.warn("writing to stdout, so no output compression (use external tool)") else: nextFile = NextFile(out_file) output = OutputSplitter(nextFile, file_size, file_compress) # process pages logging.info("Starting page extraction from %s.", input_file) extract_start = default_timer() # Parallel Map/Reduce: # - pages to be processed are dispatched to workers # - a reduce process collects the results, sort them and print them. maxsize = 10 * process_count # output queue output_queue = Queue(maxsize=maxsize) # Reduce job that sorts and prints output reduce = Process(target=reduce_process, args=(output_queue, output)) reduce.start() # initialize jobs queue jobs_queue = Queue(maxsize=maxsize) # start worker processes logging.info("Using %d extract processes.", process_count) workers = [] for _ in xrange(max(1, process_count)): extractor = Process(target=extract_process, args=(jobs_queue, output_queue)) extractor.daemon = True # only live while parent process lives extractor.start() workers.append(extractor) # Mapper process # we collect individual lines, since str.join() is significantly faster # than concatenation page = [] id = None last_id = None ordinal = 0 # page count inText = False redirect = False for line in input: line = line.decode('utf-8') if '<' not in line: # faster than doing re.search() if inText: page.append(line) continue m = tagRE.search(line) if not m: continue tag = m.group(2) if tag == 'page': page = [] redirect = False elif tag == 'id' and not id: id = m.group(3) elif tag == 'title': title = m.group(3) elif tag == 'redirect': redirect = True elif tag == 'text': inText = True line = line[m.start(3):m.end(3)] page.append(line) if m.lastindex == 4: # open-close inText = False elif tag == '/text': if m.group(1): page.append(m.group(1)) inText = False elif inText: page.append(line) elif tag == '/page': colon = title.find(':') if (colon < 0 or title[:colon] in acceptedNamespaces) and id != last_id and \ not redirect and not title.startswith(templateNamespace): job = (id, title, page, ordinal) jobs_queue.put(job) # goes to any available extract_process last_id = id ordinal += 1 id = None page = [] input.close() # signal termination for w in workers: jobs_queue.put(None) # wait for workers to terminate for w in workers: w.join() # signal end of work to reduce process output_queue.put(None) # wait for it to finish reduce.join() if output != sys.stdout: output.close() extract_duration = default_timer() - extract_start extract_rate = ordinal / extract_duration logging.info("Finished %d-process extraction of %d articles in %.1fs (%.1f art/s)", process_count, ordinal, extract_duration, extract_rate) #---------------------------------------------------------------------- # Multiprocess support def extract_process(jobs_queue, output_queue): """Pull tuples of raw page content, do CPU/regex-heavy fixup, push finished text :param job_queue: where to get jobs. :param output_queue: where to queue extracted text for output. """ while True: job = jobs_queue.get() # job is (id, title, page, ordinal) if job: out = StringIO() # memory buffer Extractor(*job[:3]).extract(out) # (id, title, page) text = out.getvalue() output_queue.put((job[3], text)) # (ordinal, extracted_text) out.close() else: break def reduce_process(output_queue, output): """Pull finished article text, write series of files (or stdout) :param output_queue: text to be output. :param output: file object where to print. """ interval_start = default_timer() period = 100000 # FIXME: use a heap ordering_buffer = {} # collected pages next_ordinal = 0 # sequence number of pages while True: if next_ordinal in ordering_buffer: output.write(ordering_buffer.pop(next_ordinal)) next_ordinal += 1 # progress report if next_ordinal % period == 0: interval_rate = period / (default_timer() - interval_start) logging.info("Extracted %d articles (%.1f art/s)", next_ordinal, interval_rate) interval_start = default_timer() else: # mapper puts None to signal finish pair = output_queue.get() if not pair: break ordinal, text = pair ordering_buffer[ordinal] = text # ---------------------------------------------------------------------- # Minimum size of output files minFileSize = 200 * 1024 def main(): global urlbase, acceptedNamespaces global expand_templates, templateCache parser = argparse.ArgumentParser(prog=os.path.basename(sys.argv[0]), formatter_class=argparse.RawDescriptionHelpFormatter, description=__doc__) parser.add_argument("input", help="XML wiki dump file") groupO = parser.add_argument_group('Output') groupO.add_argument("-o", "--output", default="text", help="directory for extracted files (or '-' for dumping to stdin)") groupO.add_argument("-b", "--bytes", default="1M", help="maximum bytes per output file (default %(default)s)", metavar="n[KMG]") groupO.add_argument("-c", "--compress", action="store_true", help="compress output files using bzip") groupP = parser.add_argument_group('Processing') groupP.add_argument("--html", action="store_true", help="produce HTML output, subsumes --links and --sections") groupP.add_argument("-l", "--links", action="store_true", help="preserve links") groupP.add_argument("-ns", "--namespaces", default="", metavar="ns1,ns2", help="accepted namespaces") groupP.add_argument("-s", "--sections", action="store_true", help="preserve sections") groupP.add_argument("--templates", help="use or create file containing templates") groupP.add_argument("--no-templates", action="store_false", help="Do not expand templates") default_process_count = cpu_count() - 1 parser.add_argument("--processes", type=int, default=default_process_count, help="Number of extract processes (default %(default)s)") groupS = parser.add_argument_group('Special') groupS.add_argument("-q", "--quiet", action="store_true", help="suppress reporting progress info") groupS.add_argument("--debug", action="store_true", help="print debug info") groupS.add_argument("-a", "--article", action="store_true", help="analyze a file containing a single article (debug option)") groupS.add_argument("-v", "--version", action="version", version='%(prog)s ' + version, help="print program version") args = parser.parse_args() Extractor.keepLinks = args.links Extractor.keepSections = args.sections Extractor.toHTML = args.html if args.html: Extractor.keepLinks = True Extractor.keepSections = True expand_templates = args.no_templates try: power = 'kmg'.find(args.bytes[-1].lower()) + 1 file_size = int(args.bytes[:-1]) * 1024 ** power if file_size < minFileSize: raise ValueError() except ValueError: logging.error('Insufficient or invalid size: %s', args.bytes) return if args.namespaces: acceptedNamespaces = set(args.namespaces.split(',')) FORMAT = '%(levelname)s: %(message)s' logging.basicConfig(format=FORMAT) logger = logging.getLogger() if not args.quiet: logger.setLevel(logging.INFO) if args.debug: logger.setLevel(logging.DEBUG) input_file = args.input if not Extractor.keepLinks: ignoreTag('a') # sharing cache of parser templates is too slow: #manager = Manager() #templateCache = manager.dict() if args.article: if args.templates: if os.path.exists(args.templates): with open(args.templates) as file: load_templates(file) with open(input_file) as file: page = file.read().decode('utf-8') m = re.search(r'(.*)', page) id = m.group(1) if m else 0 m = re.search(r'(.*)', page) if m: title = m.group(1) else: logging.error('Missing title element') return Extractor(id, title, [page]).extract(sys.stdout) return output_path = args.output if output_path != '-' and not os.path.isdir(output_path): try: os.makedirs(output_path) except: logging.error('Could not create: %s', output_path) return process_dump(input_file, args.templates, output_path, file_size, args.compress, args.processes) if __name__ == '__main__': main() vowpal-wabbit-8.5.0.dfsg1/demo/recall_tree/wikipara/makeparadata.py000077500000000000000000000025501323713302100253060ustar00rootroot00000000000000#! /usr/bin/env python # about 1.8 million documents (3x training examples) # # 11502.9 1796001 3130318 0.364573 import os import random import struct import string import sys import time import DocGenerator random.seed (90210) sys.stdout = os.fdopen (sys.stdout.fileno (), 'w', 0) sys.stderr = os.fdopen (sys.stderr.fileno (), 'w', 0) numlabels = int(sys.argv[1]) numparagraphs = int(sys.argv[2]) start = time.time () skip = 0 keep = 0 exnum = 0 with open('docid2label','wb') as f: for docid, paragraphs in DocGenerator.docs ('text/AA/wiki_00.shuf.bz2'): goodparagraphs = [n for n in range (len (paragraphs)) if len (paragraphs[n].split ()) > 20] if len (goodparagraphs) < numparagraphs: skip += 1 continue keep += 1 f.write ('%s\t%u\n'% (docid, keep)) random.shuffle (goodparagraphs) for n in goodparagraphs[0:(numparagraphs-1)]: tokens = [ t.strip (string.punctuation).translate(None,":|") for t in paragraphs[n].split () ] sys.stdout.write ("%u | %s\n"%(keep,' '.join(tokens))) for n in goodparagraphs[(numparagraphs-1):numparagraphs]: tokens = [ t.strip (string.punctuation).translate(None,":|") for t in paragraphs[n].split () ] sys.stderr.write ("%u | %s\n"%(keep,' '.join(tokens))) if keep >= numlabels: break if keep < numlabels: exit(1) vowpal-wabbit-8.5.0.dfsg1/depcomp000077500000000000000000000560161323713302100166650ustar00rootroot00000000000000#! /bin/sh # depcomp - compile a program generating dependencies as side-effects scriptversion=2013-05-30.07; # UTC # Copyright (C) 1999-2014 Free Software Foundation, Inc. # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2, or (at your option) # any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program. If not, see . # As a special exception to the GNU General Public License, if you # distribute this file as part of a program that contains a # configuration script generated by Autoconf, you may include it under # the same distribution terms that you use for the rest of that program. # Originally written by Alexandre Oliva . case $1 in '') echo "$0: No command. Try '$0 --help' for more information." 1>&2 exit 1; ;; -h | --h*) cat <<\EOF Usage: depcomp [--help] [--version] PROGRAM [ARGS] Run PROGRAMS ARGS to compile a file, generating dependencies as side-effects. Environment variables: depmode Dependency tracking mode. source Source file read by 'PROGRAMS ARGS'. object Object file output by 'PROGRAMS ARGS'. DEPDIR directory where to store dependencies. depfile Dependency file to output. tmpdepfile Temporary file to use when outputting dependencies. libtool Whether libtool is used (yes/no). Report bugs to . EOF exit $? ;; -v | --v*) echo "depcomp $scriptversion" exit $? ;; esac # Get the directory component of the given path, and save it in the # global variables '$dir'. Note that this directory component will # be either empty or ending with a '/' character. This is deliberate. set_dir_from () { case $1 in */*) dir=`echo "$1" | sed -e 's|/[^/]*$|/|'`;; *) dir=;; esac } # Get the suffix-stripped basename of the given path, and save it the # global variable '$base'. set_base_from () { base=`echo "$1" | sed -e 's|^.*/||' -e 's/\.[^.]*$//'` } # If no dependency file was actually created by the compiler invocation, # we still have to create a dummy depfile, to avoid errors with the # Makefile "include basename.Plo" scheme. make_dummy_depfile () { echo "#dummy" > "$depfile" } # Factor out some common post-processing of the generated depfile. # Requires the auxiliary global variable '$tmpdepfile' to be set. aix_post_process_depfile () { # If the compiler actually managed to produce a dependency file, # post-process it. if test -f "$tmpdepfile"; then # Each line is of the form 'foo.o: dependency.h'. # Do two passes, one to just change these to # $object: dependency.h # and one to simply output # dependency.h: # which is needed to avoid the deleted-header problem. { sed -e "s,^.*\.[$lower]*:,$object:," < "$tmpdepfile" sed -e "s,^.*\.[$lower]*:[$tab ]*,," -e 's,$,:,' < "$tmpdepfile" } > "$depfile" rm -f "$tmpdepfile" else make_dummy_depfile fi } # A tabulation character. tab=' ' # A newline character. nl=' ' # Character ranges might be problematic outside the C locale. # These definitions help. upper=ABCDEFGHIJKLMNOPQRSTUVWXYZ lower=abcdefghijklmnopqrstuvwxyz digits=0123456789 alpha=${upper}${lower} if test -z "$depmode" || test -z "$source" || test -z "$object"; then echo "depcomp: Variables source, object and depmode must be set" 1>&2 exit 1 fi # Dependencies for sub/bar.o or sub/bar.obj go into sub/.deps/bar.Po. depfile=${depfile-`echo "$object" | sed 's|[^\\/]*$|'${DEPDIR-.deps}'/&|;s|\.\([^.]*\)$|.P\1|;s|Pobj$|Po|'`} tmpdepfile=${tmpdepfile-`echo "$depfile" | sed 's/\.\([^.]*\)$/.T\1/'`} rm -f "$tmpdepfile" # Avoid interferences from the environment. gccflag= dashmflag= # Some modes work just like other modes, but use different flags. We # parameterize here, but still list the modes in the big case below, # to make depend.m4 easier to write. Note that we *cannot* use a case # here, because this file can only contain one case statement. if test "$depmode" = hp; then # HP compiler uses -M and no extra arg. gccflag=-M depmode=gcc fi if test "$depmode" = dashXmstdout; then # This is just like dashmstdout with a different argument. dashmflag=-xM depmode=dashmstdout fi cygpath_u="cygpath -u -f -" if test "$depmode" = msvcmsys; then # This is just like msvisualcpp but w/o cygpath translation. # Just convert the backslash-escaped backslashes to single forward # slashes to satisfy depend.m4 cygpath_u='sed s,\\\\,/,g' depmode=msvisualcpp fi if test "$depmode" = msvc7msys; then # This is just like msvc7 but w/o cygpath translation. # Just convert the backslash-escaped backslashes to single forward # slashes to satisfy depend.m4 cygpath_u='sed s,\\\\,/,g' depmode=msvc7 fi if test "$depmode" = xlc; then # IBM C/C++ Compilers xlc/xlC can output gcc-like dependency information. gccflag=-qmakedep=gcc,-MF depmode=gcc fi case "$depmode" in gcc3) ## gcc 3 implements dependency tracking that does exactly what ## we want. Yay! Note: for some reason libtool 1.4 doesn't like ## it if -MD -MP comes after the -MF stuff. Hmm. ## Unfortunately, FreeBSD c89 acceptance of flags depends upon ## the command line argument order; so add the flags where they ## appear in depend2.am. Note that the slowdown incurred here ## affects only configure: in makefiles, %FASTDEP% shortcuts this. for arg do case $arg in -c) set fnord "$@" -MT "$object" -MD -MP -MF "$tmpdepfile" "$arg" ;; *) set fnord "$@" "$arg" ;; esac shift # fnord shift # $arg done "$@" stat=$? if test $stat -ne 0; then rm -f "$tmpdepfile" exit $stat fi mv "$tmpdepfile" "$depfile" ;; gcc) ## Note that this doesn't just cater to obsosete pre-3.x GCC compilers. ## but also to in-use compilers like IMB xlc/xlC and the HP C compiler. ## (see the conditional assignment to $gccflag above). ## There are various ways to get dependency output from gcc. Here's ## why we pick this rather obscure method: ## - Don't want to use -MD because we'd like the dependencies to end ## up in a subdir. Having to rename by hand is ugly. ## (We might end up doing this anyway to support other compilers.) ## - The DEPENDENCIES_OUTPUT environment variable makes gcc act like ## -MM, not -M (despite what the docs say). Also, it might not be ## supported by the other compilers which use the 'gcc' depmode. ## - Using -M directly means running the compiler twice (even worse ## than renaming). if test -z "$gccflag"; then gccflag=-MD, fi "$@" -Wp,"$gccflag$tmpdepfile" stat=$? if test $stat -ne 0; then rm -f "$tmpdepfile" exit $stat fi rm -f "$depfile" echo "$object : \\" > "$depfile" # The second -e expression handles DOS-style file names with drive # letters. sed -e 's/^[^:]*: / /' \ -e 's/^['$alpha']:\/[^:]*: / /' < "$tmpdepfile" >> "$depfile" ## This next piece of magic avoids the "deleted header file" problem. ## The problem is that when a header file which appears in a .P file ## is deleted, the dependency causes make to die (because there is ## typically no way to rebuild the header). We avoid this by adding ## dummy dependencies for each header file. Too bad gcc doesn't do ## this for us directly. ## Some versions of gcc put a space before the ':'. On the theory ## that the space means something, we add a space to the output as ## well. hp depmode also adds that space, but also prefixes the VPATH ## to the object. Take care to not repeat it in the output. ## Some versions of the HPUX 10.20 sed can't process this invocation ## correctly. Breaking it into two sed invocations is a workaround. tr ' ' "$nl" < "$tmpdepfile" \ | sed -e 's/^\\$//' -e '/^$/d' -e "s|.*$object$||" -e '/:$/d' \ | sed -e 's/$/ :/' >> "$depfile" rm -f "$tmpdepfile" ;; hp) # This case exists only to let depend.m4 do its work. It works by # looking at the text of this script. This case will never be run, # since it is checked for above. exit 1 ;; sgi) if test "$libtool" = yes; then "$@" "-Wp,-MDupdate,$tmpdepfile" else "$@" -MDupdate "$tmpdepfile" fi stat=$? if test $stat -ne 0; then rm -f "$tmpdepfile" exit $stat fi rm -f "$depfile" if test -f "$tmpdepfile"; then # yes, the sourcefile depend on other files echo "$object : \\" > "$depfile" # Clip off the initial element (the dependent). Don't try to be # clever and replace this with sed code, as IRIX sed won't handle # lines with more than a fixed number of characters (4096 in # IRIX 6.2 sed, 8192 in IRIX 6.5). We also remove comment lines; # the IRIX cc adds comments like '#:fec' to the end of the # dependency line. tr ' ' "$nl" < "$tmpdepfile" \ | sed -e 's/^.*\.o://' -e 's/#.*$//' -e '/^$/ d' \ | tr "$nl" ' ' >> "$depfile" echo >> "$depfile" # The second pass generates a dummy entry for each header file. tr ' ' "$nl" < "$tmpdepfile" \ | sed -e 's/^.*\.o://' -e 's/#.*$//' -e '/^$/ d' -e 's/$/:/' \ >> "$depfile" else make_dummy_depfile fi rm -f "$tmpdepfile" ;; xlc) # This case exists only to let depend.m4 do its work. It works by # looking at the text of this script. This case will never be run, # since it is checked for above. exit 1 ;; aix) # The C for AIX Compiler uses -M and outputs the dependencies # in a .u file. In older versions, this file always lives in the # current directory. Also, the AIX compiler puts '$object:' at the # start of each line; $object doesn't have directory information. # Version 6 uses the directory in both cases. set_dir_from "$object" set_base_from "$object" if test "$libtool" = yes; then tmpdepfile1=$dir$base.u tmpdepfile2=$base.u tmpdepfile3=$dir.libs/$base.u "$@" -Wc,-M else tmpdepfile1=$dir$base.u tmpdepfile2=$dir$base.u tmpdepfile3=$dir$base.u "$@" -M fi stat=$? if test $stat -ne 0; then rm -f "$tmpdepfile1" "$tmpdepfile2" "$tmpdepfile3" exit $stat fi for tmpdepfile in "$tmpdepfile1" "$tmpdepfile2" "$tmpdepfile3" do test -f "$tmpdepfile" && break done aix_post_process_depfile ;; tcc) # tcc (Tiny C Compiler) understand '-MD -MF file' since version 0.9.26 # FIXME: That version still under development at the moment of writing. # Make that this statement remains true also for stable, released # versions. # It will wrap lines (doesn't matter whether long or short) with a # trailing '\', as in: # # foo.o : \ # foo.c \ # foo.h \ # # It will put a trailing '\' even on the last line, and will use leading # spaces rather than leading tabs (at least since its commit 0394caf7 # "Emit spaces for -MD"). "$@" -MD -MF "$tmpdepfile" stat=$? if test $stat -ne 0; then rm -f "$tmpdepfile" exit $stat fi rm -f "$depfile" # Each non-empty line is of the form 'foo.o : \' or ' dep.h \'. # We have to change lines of the first kind to '$object: \'. sed -e "s|.*:|$object :|" < "$tmpdepfile" > "$depfile" # And for each line of the second kind, we have to emit a 'dep.h:' # dummy dependency, to avoid the deleted-header problem. sed -n -e 's|^ *\(.*\) *\\$|\1:|p' < "$tmpdepfile" >> "$depfile" rm -f "$tmpdepfile" ;; ## The order of this option in the case statement is important, since the ## shell code in configure will try each of these formats in the order ## listed in this file. A plain '-MD' option would be understood by many ## compilers, so we must ensure this comes after the gcc and icc options. pgcc) # Portland's C compiler understands '-MD'. # Will always output deps to 'file.d' where file is the root name of the # source file under compilation, even if file resides in a subdirectory. # The object file name does not affect the name of the '.d' file. # pgcc 10.2 will output # foo.o: sub/foo.c sub/foo.h # and will wrap long lines using '\' : # foo.o: sub/foo.c ... \ # sub/foo.h ... \ # ... set_dir_from "$object" # Use the source, not the object, to determine the base name, since # that's sadly what pgcc will do too. set_base_from "$source" tmpdepfile=$base.d # For projects that build the same source file twice into different object # files, the pgcc approach of using the *source* file root name can cause # problems in parallel builds. Use a locking strategy to avoid stomping on # the same $tmpdepfile. lockdir=$base.d-lock trap " echo '$0: caught signal, cleaning up...' >&2 rmdir '$lockdir' exit 1 " 1 2 13 15 numtries=100 i=$numtries while test $i -gt 0; do # mkdir is a portable test-and-set. if mkdir "$lockdir" 2>/dev/null; then # This process acquired the lock. "$@" -MD stat=$? # Release the lock. rmdir "$lockdir" break else # If the lock is being held by a different process, wait # until the winning process is done or we timeout. while test -d "$lockdir" && test $i -gt 0; do sleep 1 i=`expr $i - 1` done fi i=`expr $i - 1` done trap - 1 2 13 15 if test $i -le 0; then echo "$0: failed to acquire lock after $numtries attempts" >&2 echo "$0: check lockdir '$lockdir'" >&2 exit 1 fi if test $stat -ne 0; then rm -f "$tmpdepfile" exit $stat fi rm -f "$depfile" # Each line is of the form `foo.o: dependent.h', # or `foo.o: dep1.h dep2.h \', or ` dep3.h dep4.h \'. # Do two passes, one to just change these to # `$object: dependent.h' and one to simply `dependent.h:'. sed "s,^[^:]*:,$object :," < "$tmpdepfile" > "$depfile" # Some versions of the HPUX 10.20 sed can't process this invocation # correctly. Breaking it into two sed invocations is a workaround. sed 's,^[^:]*: \(.*\)$,\1,;s/^\\$//;/^$/d;/:$/d' < "$tmpdepfile" \ | sed -e 's/$/ :/' >> "$depfile" rm -f "$tmpdepfile" ;; hp2) # The "hp" stanza above does not work with aCC (C++) and HP's ia64 # compilers, which have integrated preprocessors. The correct option # to use with these is +Maked; it writes dependencies to a file named # 'foo.d', which lands next to the object file, wherever that # happens to be. # Much of this is similar to the tru64 case; see comments there. set_dir_from "$object" set_base_from "$object" if test "$libtool" = yes; then tmpdepfile1=$dir$base.d tmpdepfile2=$dir.libs/$base.d "$@" -Wc,+Maked else tmpdepfile1=$dir$base.d tmpdepfile2=$dir$base.d "$@" +Maked fi stat=$? if test $stat -ne 0; then rm -f "$tmpdepfile1" "$tmpdepfile2" exit $stat fi for tmpdepfile in "$tmpdepfile1" "$tmpdepfile2" do test -f "$tmpdepfile" && break done if test -f "$tmpdepfile"; then sed -e "s,^.*\.[$lower]*:,$object:," "$tmpdepfile" > "$depfile" # Add 'dependent.h:' lines. sed -ne '2,${ s/^ *// s/ \\*$// s/$/:/ p }' "$tmpdepfile" >> "$depfile" else make_dummy_depfile fi rm -f "$tmpdepfile" "$tmpdepfile2" ;; tru64) # The Tru64 compiler uses -MD to generate dependencies as a side # effect. 'cc -MD -o foo.o ...' puts the dependencies into 'foo.o.d'. # At least on Alpha/Redhat 6.1, Compaq CCC V6.2-504 seems to put # dependencies in 'foo.d' instead, so we check for that too. # Subdirectories are respected. set_dir_from "$object" set_base_from "$object" if test "$libtool" = yes; then # Libtool generates 2 separate objects for the 2 libraries. These # two compilations output dependencies in $dir.libs/$base.o.d and # in $dir$base.o.d. We have to check for both files, because # one of the two compilations can be disabled. We should prefer # $dir$base.o.d over $dir.libs/$base.o.d because the latter is # automatically cleaned when .libs/ is deleted, while ignoring # the former would cause a distcleancheck panic. tmpdepfile1=$dir$base.o.d # libtool 1.5 tmpdepfile2=$dir.libs/$base.o.d # Likewise. tmpdepfile3=$dir.libs/$base.d # Compaq CCC V6.2-504 "$@" -Wc,-MD else tmpdepfile1=$dir$base.d tmpdepfile2=$dir$base.d tmpdepfile3=$dir$base.d "$@" -MD fi stat=$? if test $stat -ne 0; then rm -f "$tmpdepfile1" "$tmpdepfile2" "$tmpdepfile3" exit $stat fi for tmpdepfile in "$tmpdepfile1" "$tmpdepfile2" "$tmpdepfile3" do test -f "$tmpdepfile" && break done # Same post-processing that is required for AIX mode. aix_post_process_depfile ;; msvc7) if test "$libtool" = yes; then showIncludes=-Wc,-showIncludes else showIncludes=-showIncludes fi "$@" $showIncludes > "$tmpdepfile" stat=$? grep -v '^Note: including file: ' "$tmpdepfile" if test $stat -ne 0; then rm -f "$tmpdepfile" exit $stat fi rm -f "$depfile" echo "$object : \\" > "$depfile" # The first sed program below extracts the file names and escapes # backslashes for cygpath. The second sed program outputs the file # name when reading, but also accumulates all include files in the # hold buffer in order to output them again at the end. This only # works with sed implementations that can handle large buffers. sed < "$tmpdepfile" -n ' /^Note: including file: *\(.*\)/ { s//\1/ s/\\/\\\\/g p }' | $cygpath_u | sort -u | sed -n ' s/ /\\ /g s/\(.*\)/'"$tab"'\1 \\/p s/.\(.*\) \\/\1:/ H $ { s/.*/'"$tab"'/ G p }' >> "$depfile" echo >> "$depfile" # make sure the fragment doesn't end with a backslash rm -f "$tmpdepfile" ;; msvc7msys) # This case exists only to let depend.m4 do its work. It works by # looking at the text of this script. This case will never be run, # since it is checked for above. exit 1 ;; #nosideeffect) # This comment above is used by automake to tell side-effect # dependency tracking mechanisms from slower ones. dashmstdout) # Important note: in order to support this mode, a compiler *must* # always write the preprocessed file to stdout, regardless of -o. "$@" || exit $? # Remove the call to Libtool. if test "$libtool" = yes; then while test "X$1" != 'X--mode=compile'; do shift done shift fi # Remove '-o $object'. IFS=" " for arg do case $arg in -o) shift ;; $object) shift ;; *) set fnord "$@" "$arg" shift # fnord shift # $arg ;; esac done test -z "$dashmflag" && dashmflag=-M # Require at least two characters before searching for ':' # in the target name. This is to cope with DOS-style filenames: # a dependency such as 'c:/foo/bar' could be seen as target 'c' otherwise. "$@" $dashmflag | sed "s|^[$tab ]*[^:$tab ][^:][^:]*:[$tab ]*|$object: |" > "$tmpdepfile" rm -f "$depfile" cat < "$tmpdepfile" > "$depfile" # Some versions of the HPUX 10.20 sed can't process this sed invocation # correctly. Breaking it into two sed invocations is a workaround. tr ' ' "$nl" < "$tmpdepfile" \ | sed -e 's/^\\$//' -e '/^$/d' -e '/:$/d' \ | sed -e 's/$/ :/' >> "$depfile" rm -f "$tmpdepfile" ;; dashXmstdout) # This case only exists to satisfy depend.m4. It is never actually # run, as this mode is specially recognized in the preamble. exit 1 ;; makedepend) "$@" || exit $? # Remove any Libtool call if test "$libtool" = yes; then while test "X$1" != 'X--mode=compile'; do shift done shift fi # X makedepend shift cleared=no eat=no for arg do case $cleared in no) set ""; shift cleared=yes ;; esac if test $eat = yes; then eat=no continue fi case "$arg" in -D*|-I*) set fnord "$@" "$arg"; shift ;; # Strip any option that makedepend may not understand. Remove # the object too, otherwise makedepend will parse it as a source file. -arch) eat=yes ;; -*|$object) ;; *) set fnord "$@" "$arg"; shift ;; esac done obj_suffix=`echo "$object" | sed 's/^.*\././'` touch "$tmpdepfile" ${MAKEDEPEND-makedepend} -o"$obj_suffix" -f"$tmpdepfile" "$@" rm -f "$depfile" # makedepend may prepend the VPATH from the source file name to the object. # No need to regex-escape $object, excess matching of '.' is harmless. sed "s|^.*\($object *:\)|\1|" "$tmpdepfile" > "$depfile" # Some versions of the HPUX 10.20 sed can't process the last invocation # correctly. Breaking it into two sed invocations is a workaround. sed '1,2d' "$tmpdepfile" \ | tr ' ' "$nl" \ | sed -e 's/^\\$//' -e '/^$/d' -e '/:$/d' \ | sed -e 's/$/ :/' >> "$depfile" rm -f "$tmpdepfile" "$tmpdepfile".bak ;; cpp) # Important note: in order to support this mode, a compiler *must* # always write the preprocessed file to stdout. "$@" || exit $? # Remove the call to Libtool. if test "$libtool" = yes; then while test "X$1" != 'X--mode=compile'; do shift done shift fi # Remove '-o $object'. IFS=" " for arg do case $arg in -o) shift ;; $object) shift ;; *) set fnord "$@" "$arg" shift # fnord shift # $arg ;; esac done "$@" -E \ | sed -n -e '/^# [0-9][0-9]* "\([^"]*\)".*/ s:: \1 \\:p' \ -e '/^#line [0-9][0-9]* "\([^"]*\)".*/ s:: \1 \\:p' \ | sed '$ s: \\$::' > "$tmpdepfile" rm -f "$depfile" echo "$object : \\" > "$depfile" cat < "$tmpdepfile" >> "$depfile" sed < "$tmpdepfile" '/^$/d;s/^ //;s/ \\$//;s/$/ :/' >> "$depfile" rm -f "$tmpdepfile" ;; msvisualcpp) # Important note: in order to support this mode, a compiler *must* # always write the preprocessed file to stdout. "$@" || exit $? # Remove the call to Libtool. if test "$libtool" = yes; then while test "X$1" != 'X--mode=compile'; do shift done shift fi IFS=" " for arg do case "$arg" in -o) shift ;; $object) shift ;; "-Gm"|"/Gm"|"-Gi"|"/Gi"|"-ZI"|"/ZI") set fnord "$@" shift shift ;; *) set fnord "$@" "$arg" shift shift ;; esac done "$@" -E 2>/dev/null | sed -n '/^#line [0-9][0-9]* "\([^"]*\)"/ s::\1:p' | $cygpath_u | sort -u > "$tmpdepfile" rm -f "$depfile" echo "$object : \\" > "$depfile" sed < "$tmpdepfile" -n -e 's% %\\ %g' -e '/^\(.*\)$/ s::'"$tab"'\1 \\:p' >> "$depfile" echo "$tab" >> "$depfile" sed < "$tmpdepfile" -n -e 's% %\\ %g' -e '/^\(.*\)$/ s::\1\::p' >> "$depfile" rm -f "$tmpdepfile" ;; msvcmsys) # This case exists only to let depend.m4 do its work. It works by # looking at the text of this script. This case will never be run, # since it is checked for above. exit 1 ;; none) exec "$@" ;; *) echo "Unknown depmode $depmode" 1>&2 exit 1 ;; esac exit 0 # Local Variables: # mode: shell-script # sh-indentation: 2 # eval: (add-hook 'write-file-hooks 'time-stamp) # time-stamp-start: "scriptversion=" # time-stamp-format: "%:y-%02m-%02d.%02H" # time-stamp-time-zone: "UTC" # time-stamp-end: "; # UTC" # End: vowpal-wabbit-8.5.0.dfsg1/deploy_vw/000077500000000000000000000000001323713302100173105ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/deploy_vw/deploy_vw.vcxproj000066400000000000000000000405101323713302100227350ustar00rootroot00000000000000 DebugLeakCheck Win32 DebugLeakCheck x64 Debug Win32 Debug x64 Release Win32 Release x64 {D3D9B744-D0FC-4BC7-94A8-89C1AC5692DE} deploy_vw ..\vowpalwabbit\ ..\sdl\SDL-7.0-Recommended.ruleset true Application true v140 MultiByte Application true v140 MultiByte Application true v140 MultiByte Application true v140 MultiByte Application false v140 true MultiByte Application false v140 true MultiByte $(SolutionDir)deploy\$(PlatformShortName)\$(Configuration)\ $(SolutionDir)deploy\stage\$(PlatformShortName)\$(Configuration)\ Clean $(SolutionDir)deploy\$(PlatformShortName)\$(Configuration)\ $(SolutionDir)deploy\stage\$(PlatformShortName)\$(Configuration)\ Clean $(SolutionDir)deploy\$(PlatformShortName)\$(Configuration)\ $(SolutionDir)deploy\stage\$(PlatformShortName)\$(Configuration)\ Clean $(SolutionDir)deploy\$(PlatformShortName)\$(Configuration)\ $(SolutionDir)deploy\stage\$(PlatformShortName)\$(Configuration)\ Clean $(SolutionDir)deploy\$(PlatformShortName)\$(Configuration)\ $(SolutionDir)deploy\stage\$(PlatformShortName)\$(Configuration)\ Clean $(SolutionDir)deploy\$(PlatformShortName)\$(Configuration)\ $(SolutionDir)deploy\stage\$(PlatformShortName)\$(Configuration)\ Clean Level3 Disabled true true rd /s /y $(OutputPath) Cleaning Deploy xcopy /f /v /i /r /y "$(SolutionDir)$(PlatformShortName)\$(Configuration)\vw.exe" "$(OutputPath)" xcopy /f /v /i /r /y "$(SolutionDir)$(PlatformShortName)\$(Configuration)\vw.pdb" "$(OutputPath)" xcopy /f /v /i /r /y "$(SolutionDir)$(PlatformShortName)\$(Configuration)\spanning_tree.exe" "$(OutputPath)" xcopy /f /v /i /r /y "$(SolutionDir)$(PlatformShortName)\$(Configuration)\spanning_tree.pdb" "$(OutputPath)" xcopy /v /i /r /y "$(VCInstallDir)\redist\Debug_NonRedist\$(PlatformShortName)\Microsoft.VC120.DebugCRT" "$(OutputPath)" xcopy /v /i /r /y "%$(VCInstallDir)redist\Debug_NonRedist\$(PlatformShortName)\Microsoft.VC120.DebugCXXAMP" "$(OutputPath)" xcopy /v /i /r /y "$(VCInstallDir)redist\Debug_NonRedist\$(PlatformShortName)\Microsoft.VC120.DebugOPENMP" "$(OutputPath)" Level3 Disabled true true rd /s /y $(OutputPath) Cleaning Deploy xcopy /f /v /i /r /y "$(SolutionDir)$(PlatformShortName)\$(Configuration)\vw.exe" "$(OutputPath)" xcopy /f /v /i /r /y "$(SolutionDir)$(PlatformShortName)\$(Configuration)\vw.pdb" "$(OutputPath)" xcopy /f /v /i /r /y "$(SolutionDir)$(PlatformShortName)\$(Configuration)\spanning_tree.exe" "$(OutputPath)" xcopy /f /v /i /r /y "$(SolutionDir)$(PlatformShortName)\$(Configuration)\spanning_tree.pdb" "$(OutputPath)" xcopy /v /i /r /y "$(VCInstallDir)\redist\Debug_NonRedist\$(PlatformShortName)\Microsoft.VC120.DebugCRT" "$(OutputPath)" xcopy /v /i /r /y "%$(VCInstallDir)redist\Debug_NonRedist\$(PlatformShortName)\Microsoft.VC120.DebugCXXAMP" "$(OutputPath)" xcopy /v /i /r /y "$(VCInstallDir)redist\Debug_NonRedist\$(PlatformShortName)\Microsoft.VC120.DebugOPENMP" "$(OutputPath)" Level3 Disabled true true rd /s /y $(OutputPath) Cleaning Deploy xcopy /f /v /i /r /y "$(SolutionDir)$(PlatformShortName)\$(Configuration)\vw.exe" "$(OutputPath)" xcopy /f /v /i /r /y "$(SolutionDir)$(PlatformShortName)\$(Configuration)\vw.pdb" "$(OutputPath)" xcopy /f /v /i /r /y "$(SolutionDir)$(PlatformShortName)\$(Configuration)\spanning_tree.exe" "$(OutputPath)" xcopy /f /v /i /r /y "$(SolutionDir)$(PlatformShortName)\$(Configuration)\spanning_tree.pdb" "$(OutputPath)" xcopy /v /i /r /y "$(VCInstallDir)\redist\Debug_NonRedist\$(PlatformShortName)\Microsoft.VC120.DebugCRT" "$(OutputPath)" xcopy /v /i /r /y "%$(VCInstallDir)redist\Debug_NonRedist\$(PlatformShortName)\Microsoft.VC120.DebugCXXAMP" "$(OutputPath)" xcopy /v /i /r /y "$(VCInstallDir)redist\Debug_NonRedist\$(PlatformShortName)\Microsoft.VC120.DebugOPENMP" "$(OutputPath)" Level3 Disabled true true rd /s /y $(OutputPath) Cleaning Deploy xcopy /f /v /i /r /y "$(SolutionDir)$(PlatformShortName)\$(Configuration)\vw.exe" "$(OutputPath)" xcopy /f /v /i /r /y "$(SolutionDir)$(PlatformShortName)\$(Configuration)\vw.pdb" "$(OutputPath)" xcopy /f /v /i /r /y "$(SolutionDir)$(PlatformShortName)\$(Configuration)\spanning_tree.exe" "$(OutputPath)" xcopy /f /v /i /r /y "$(SolutionDir)$(PlatformShortName)\$(Configuration)\spanning_tree.pdb" "$(OutputPath)" xcopy /v /i /r /y "$(VCInstallDir)\redist\Debug_NonRedist\$(PlatformShortName)\Microsoft.VC120.DebugCRT" "$(OutputPath)" xcopy /v /i /r /y "%$(VCInstallDir)redist\Debug_NonRedist\$(PlatformShortName)\Microsoft.VC120.DebugCXXAMP" "$(OutputPath)" xcopy /v /i /r /y "$(VCInstallDir)redist\Debug_NonRedist\$(PlatformShortName)\Microsoft.VC120.DebugOPENMP" "$(OutputPath)" Level3 MaxSpeed true true true true true true rd /s /y $(OutputPath) Cleaning Deploy xcopy /f /v /i /r /y "$(SolutionDir)$(PlatformShortName)\$(Configuration)\vw.exe" "$(OutputPath)" xcopy /f /v /i /r /y "$(SolutionDir)$(PlatformShortName)\$(Configuration)\spanning_tree.exe" "$(OutputPath)" xcopy /v /i /r /y "$(VCInstallDir)\redist\$(PlatformShortName)\Microsoft.VC120.CRT" "$(OutputPath)" xcopy /v /i /r /y "%$(VCInstallDir)redist\$(PlatformShortName)\Microsoft.VC120.CXXAMP" "$(OutputPath)" xcopy /v /i /r /y "$(VCInstallDir)redist\$(PlatformShortName)\Microsoft.VC120.OPENMP" "$(OutputPath)" Level3 MaxSpeed true true true true true true rd /s /y $(OutputPath) Cleaning Deploy xcopy /f /v /i /r /y "$(SolutionDir)$(PlatformShortName)\$(Configuration)\vw.exe" "$(OutputPath)" xcopy /f /v /i /r /y "$(SolutionDir)$(PlatformShortName)\$(Configuration)\spanning_tree.exe" "$(OutputPath)" xcopy /v /i /r /y "$(VCInstallDir)\redist\$(PlatformShortName)\Microsoft.VC120.CRT" "$(OutputPath)" xcopy /v /i /r /y "%$(VCInstallDir)redist\$(PlatformShortName)\Microsoft.VC120.CXXAMP" "$(OutputPath)" xcopy /v /i /r /y "$(VCInstallDir)redist\$(PlatformShortName)\Microsoft.VC120.OPENMP" "$(OutputPath)" This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}. vowpal-wabbit-8.5.0.dfsg1/deployvw.bat000066400000000000000000000030241323713302100176400ustar00rootroot00000000000000@echo off if "%1" == "" goto usage if not "%2" == "" goto usage if "%VSINSTALLDIR%" == "" goto missing if /i %1 == x86 goto x86 if /i %1 == x64 goto x64 if %VSINSTALLDIR% :x86 devenv vowpalwabbit\vw.sln /Clean "Release|x86" devenv vowpalwabbit\vw.sln /Rebuild "Release|x86" xcopy /v /i /r /y vowpalwabbit\x86\Release\vw.exe deploy\x86\Release\ xcopy /v /i /r /y vowpalwabbit\x86\Release\spanning_tree.exe deploy\x86\Release\ xcopy /v /i /r /y "%VSINSTALLDIR%VC\redist\x86\Microsoft.VC120.CRT" deploy\x86\Release\ xcopy /v /i /r /y "%VSINSTALLDIR%VC\redist\x86\Microsoft.VC120.CXXAMP" deploy\x86\Release\ xcopy /v /i /r /y "%VSINSTALLDIR%VC\redist\x86\Microsoft.VC120.OPENMP" deploy\x86\Release\ goto :eof :x64 devenv vowpalwabbit\vw.sln /Clean "Release|x64" devenv vowpalwabbit\vw.sln /Rebuild "Release|x64" xcopy /v /i /r /y vowpalwabbit\x64\Release\vw.exe deploy\x64\Release\ xcopy /v /i /r /y vowpalwabbit\x64\Release\spanning_tree.exe deploy\x64\Release\ xcopy /v /i /r /y "%VSINSTALLDIR%VC\redist\x64\Microsoft.VC120.CRT" deploy\x64\Release\ xcopy /v /i /r /y "%VSINSTALLDIR%VC\redist\x64\Microsoft.VC120.CXXAMP" deploy\x64\Release\ xcopy /v /i /r /y "%VSINSTALLDIR%VC\redist\x64\Microsoft.VC120.OPENMP" deploy\x64\Release\ goto :eof :usage echo Error in script usage. The correct usage is: echo %0 [option] echo where [option] is: x86 ^| x64 echo: echo For example: echo %0 x86 goto :eof :missing echo The variable "VSINSTALLDIR" is missing. Visual Studio 2013 might not be installed. goto :eof :eof vowpal-wabbit-8.5.0.dfsg1/doc/000077500000000000000000000000001323713302100160455ustar00rootroot00000000000000vowpal-wabbit-8.5.0.dfsg1/doc/Doxyfile000066400000000000000000003036331323713302100175630ustar00rootroot00000000000000# Doxyfile 1.8.6 # This file describes the settings to be used by the documentation system # doxygen (www.doxygen.org) for a project. # # All text after a double hash (##) is considered a comment and is placed in # front of the TAG it is preceding. # # All text after a single hash (#) is considered a comment and will be ignored. # The format is: # TAG = value [value, ...] # For lists, items can also be appended using: # TAG += value [value, ...] # Values that contain spaces should be placed between quotes (\" \"). #--------------------------------------------------------------------------- # Project related configuration options #--------------------------------------------------------------------------- # This tag specifies the encoding used for all characters in the config file # that follow. The default is UTF-8 which is also the encoding used for all text # before the first occurrence of this tag. Doxygen uses libiconv (or the iconv # built into libc) for the transcoding. See http://www.gnu.org/software/libiconv # for the list of possible encodings. # The default value is: UTF-8. DOXYFILE_ENCODING = UTF-8 # The PROJECT_NAME tag is a single word (or a sequence of words surrounded by # double-quotes, unless you are using Doxywizard) that should identify the # project for which the documentation is generated. This name is used in the # title of most generated pages and in a few other places. # The default value is: My Project. PROJECT_NAME = "Vowpal Wabbit" # The PROJECT_NUMBER tag can be used to enter a project or revision number. This # could be handy for archiving the generated documentation or if some version # control system is used. PROJECT_NUMBER = # Using the PROJECT_BRIEF tag one can provide an optional one line description # for a project that appears at the top of each page and should give viewer a # quick idea about the purpose of the project. Keep the description short. PROJECT_BRIEF = # With the PROJECT_LOGO tag one can specify an logo or icon that is included in # the documentation. The maximum height of the logo should not exceed 55 pixels # and the maximum width should not exceed 200 pixels. Doxygen will copy the logo # to the output directory. PROJECT_LOGO = # The OUTPUT_DIRECTORY tag is used to specify the (relative or absolute) path # into which the generated documentation will be written. If a relative path is # entered, it will be relative to the location where doxygen was started. If # left blank the current directory will be used. OUTPUT_DIRECTORY = ./ # If the CREATE_SUBDIRS tag is set to YES, then doxygen will create 4096 sub- # directories (in 2 levels) under the output directory of each output format and # will distribute the generated files over these directories. Enabling this # option can be useful when feeding doxygen a huge amount of source files, where # putting all generated files in the same directory would otherwise causes # performance problems for the file system. # The default value is: NO. CREATE_SUBDIRS = NO # The OUTPUT_LANGUAGE tag is used to specify the language in which all # documentation generated by doxygen is written. Doxygen will use this # information to generate all constant output in the proper language. # Possible values are: Afrikaans, Arabic, Armenian, Brazilian, Catalan, Chinese, # Chinese-Traditional, Croatian, Czech, Danish, Dutch, English (United States), # Esperanto, Farsi (Persian), Finnish, French, German, Greek, Hungarian, # Indonesian, Italian, Japanese, Japanese-en (Japanese with English messages), # Korean, Korean-en (Korean with English messages), Latvian, Lithuanian, # Macedonian, Norwegian, Persian (Farsi), Polish, Portuguese, Romanian, Russian, # Serbian, Serbian-Cyrillic, Slovak, Slovene, Spanish, Swedish, Turkish, # Ukrainian and Vietnamese. # The default value is: English. OUTPUT_LANGUAGE = English # If the BRIEF_MEMBER_DESC tag is set to YES doxygen will include brief member # descriptions after the members that are listed in the file and class # documentation (similar to Javadoc). Set to NO to disable this. # The default value is: YES. BRIEF_MEMBER_DESC = YES # If the REPEAT_BRIEF tag is set to YES doxygen will prepend the brief # description of a member or function before the detailed description # # Note: If both HIDE_UNDOC_MEMBERS and BRIEF_MEMBER_DESC are set to NO, the # brief descriptions will be completely suppressed. # The default value is: YES. REPEAT_BRIEF = YES # This tag implements a quasi-intelligent brief description abbreviator that is # used to form the text in various listings. Each string in this list, if found # as the leading text of the brief description, will be stripped from the text # and the result, after processing the whole list, is used as the annotated # text. Otherwise, the brief description is used as-is. If left blank, the # following values are used ($name is automatically replaced with the name of # the entity):The $name class, The $name widget, The $name file, is, provides, # specifies, contains, represents, a, an and the. ABBREVIATE_BRIEF = # If the ALWAYS_DETAILED_SEC and REPEAT_BRIEF tags are both set to YES then # doxygen will generate a detailed section even if there is only a brief # description. # The default value is: NO. ALWAYS_DETAILED_SEC = NO # If the INLINE_INHERITED_MEMB tag is set to YES, doxygen will show all # inherited members of a class in the documentation of that class as if those # members were ordinary class members. Constructors, destructors and assignment # operators of the base classes will not be shown. # The default value is: NO. INLINE_INHERITED_MEMB = NO # If the FULL_PATH_NAMES tag is set to YES doxygen will prepend the full path # before files name in the file list and in the header files. If set to NO the # shortest path that makes the file name unique will be used # The default value is: YES. FULL_PATH_NAMES = YES # The STRIP_FROM_PATH tag can be used to strip a user-defined part of the path. # Stripping is only done if one of the specified strings matches the left-hand # part of the path. The tag can be used to show relative paths in the file list. # If left blank the directory from which doxygen is run is used as the path to # strip. # # Note that you can specify absolute paths here, but also relative paths, which # will be relative from the directory where doxygen is started. # This tag requires that the tag FULL_PATH_NAMES is set to YES. STRIP_FROM_PATH = # The STRIP_FROM_INC_PATH tag can be used to strip a user-defined part of the # path mentioned in the documentation of a class, which tells the reader which # header file to include in order to use a class. If left blank only the name of # the header file containing the class definition is used. Otherwise one should # specify the list of include paths that are normally passed to the compiler # using the -I flag. STRIP_FROM_INC_PATH = # If the SHORT_NAMES tag is set to YES, doxygen will generate much shorter (but # less readable) file names. This can be useful is your file systems doesn't # support long names like on DOS, Mac, or CD-ROM. # The default value is: NO. SHORT_NAMES = NO # If the JAVADOC_AUTOBRIEF tag is set to YES then doxygen will interpret the # first line (until the first dot) of a Javadoc-style comment as the brief # description. If set to NO, the Javadoc-style will behave just like regular Qt- # style comments (thus requiring an explicit @brief command for a brief # description.) # The default value is: NO. JAVADOC_AUTOBRIEF = NO # If the QT_AUTOBRIEF tag is set to YES then doxygen will interpret the first # line (until the first dot) of a Qt-style comment as the brief description. If # set to NO, the Qt-style will behave just like regular Qt-style comments (thus # requiring an explicit \brief command for a brief description.) # The default value is: NO. QT_AUTOBRIEF = NO # The MULTILINE_CPP_IS_BRIEF tag can be set to YES to make doxygen treat a # multi-line C++ special comment block (i.e. a block of //! or /// comments) as # a brief description. This used to be the default behavior. The new default is # to treat a multi-line C++ comment block as a detailed description. Set this # tag to YES if you prefer the old behavior instead. # # Note that setting this tag to YES also means that rational rose comments are # not recognized any more. # The default value is: NO. MULTILINE_CPP_IS_BRIEF = NO # If the INHERIT_DOCS tag is set to YES then an undocumented member inherits the # documentation from any documented member that it re-implements. # The default value is: YES. INHERIT_DOCS = YES # If the SEPARATE_MEMBER_PAGES tag is set to YES, then doxygen will produce a # new page for each member. If set to NO, the documentation of a member will be # part of the file/class/namespace that contains it. # The default value is: NO. SEPARATE_MEMBER_PAGES = NO # The TAB_SIZE tag can be used to set the number of spaces in a tab. Doxygen # uses this value to replace tabs by spaces in code fragments. # Minimum value: 1, maximum value: 16, default value: 4. TAB_SIZE = 4 # This tag can be used to specify a number of aliases that act as commands in # the documentation. An alias has the form: # name=value # For example adding # "sideeffect=@par Side Effects:\n" # will allow you to put the command \sideeffect (or @sideeffect) in the # documentation, which will result in a user-defined paragraph with heading # "Side Effects:". You can put \n's in the value part of an alias to insert # newlines. ALIASES = # This tag can be used to specify a number of word-keyword mappings (TCL only). # A mapping has the form "name=value". For example adding "class=itcl::class" # will allow you to use the command class in the itcl::class meaning. TCL_SUBST = # Set the OPTIMIZE_OUTPUT_FOR_C tag to YES if your project consists of C sources # only. Doxygen will then generate output that is more tailored for C. For # instance, some of the names that are used will be different. The list of all # members will be omitted, etc. # The default value is: NO. OPTIMIZE_OUTPUT_FOR_C = NO # Set the OPTIMIZE_OUTPUT_JAVA tag to YES if your project consists of Java or # Python sources only. Doxygen will then generate output that is more tailored # for that language. For instance, namespaces will be presented as packages, # qualified scopes will look different, etc. # The default value is: NO. OPTIMIZE_OUTPUT_JAVA = NO # Set the OPTIMIZE_FOR_FORTRAN tag to YES if your project consists of Fortran # sources. Doxygen will then generate output that is tailored for Fortran. # The default value is: NO. OPTIMIZE_FOR_FORTRAN = NO # Set the OPTIMIZE_OUTPUT_VHDL tag to YES if your project consists of VHDL # sources. Doxygen will then generate output that is tailored for VHDL. # The default value is: NO. OPTIMIZE_OUTPUT_VHDL = NO # Doxygen selects the parser to use depending on the extension of the files it # parses. With this tag you can assign which parser to use for a given # extension. Doxygen has a built-in mapping, but you can override or extend it # using this tag. The format is ext=language, where ext is a file extension, and # language is one of the parsers supported by doxygen: IDL, Java, Javascript, # C#, C, C++, D, PHP, Objective-C, Python, Fortran, VHDL. For instance to make # doxygen treat .inc files as Fortran files (default is PHP), and .f files as C # (default is Fortran), use: inc=Fortran f=C. # # Note For files without extension you can use no_extension as a placeholder. # # Note that for custom extensions you also need to set FILE_PATTERNS otherwise # the files are not read by doxygen. EXTENSION_MAPPING = # If the MARKDOWN_SUPPORT tag is enabled then doxygen pre-processes all comments # according to the Markdown format, which allows for more readable # documentation. See http://daringfireball.net/projects/markdown/ for details. # The output of markdown processing is further processed by doxygen, so you can # mix doxygen, HTML, and XML commands with Markdown formatting. Disable only in # case of backward compatibilities issues. # The default value is: YES. MARKDOWN_SUPPORT = YES # When enabled doxygen tries to link words that correspond to documented # classes, or namespaces to their corresponding documentation. Such a link can # be prevented in individual cases by by putting a % sign in front of the word # or globally by setting AUTOLINK_SUPPORT to NO. # The default value is: YES. AUTOLINK_SUPPORT = YES # If you use STL classes (i.e. std::string, std::vector, etc.) but do not want # to include (a tag file for) the STL sources as input, then you should set this # tag to YES in order to let doxygen match functions declarations and # definitions whose arguments contain STL classes (e.g. func(std::string); # versus func(std::string) {}). This also make the inheritance and collaboration # diagrams that involve STL classes more complete and accurate. # The default value is: NO. BUILTIN_STL_SUPPORT = NO # If you use Microsoft's C++/CLI language, you should set this option to YES to # enable parsing support. # The default value is: NO. CPP_CLI_SUPPORT = YES # Set the SIP_SUPPORT tag to YES if your project consists of sip (see: # http://www.riverbankcomputing.co.uk/software/sip/intro) sources only. Doxygen # will parse them like normal C++ but will assume all classes use public instead # of private inheritance when no explicit protection keyword is present. # The default value is: NO. SIP_SUPPORT = NO # For Microsoft's IDL there are propget and propput attributes to indicate # getter and setter methods for a property. Setting this option to YES will make # doxygen to replace the get and set methods by a property in the documentation. # This will only work if the methods are indeed getting or setting a simple # type. If this is not the case, or you want to show the methods anyway, you # should set this option to NO. # The default value is: YES. IDL_PROPERTY_SUPPORT = YES # If member grouping is used in the documentation and the DISTRIBUTE_GROUP_DOC # tag is set to YES, then doxygen will reuse the documentation of the first # member in the group (if any) for the other members of the group. By default # all members of a group must be documented explicitly. # The default value is: NO. DISTRIBUTE_GROUP_DOC = NO # Set the SUBGROUPING tag to YES to allow class member groups of the same type # (for instance a group of public functions) to be put as a subgroup of that # type (e.g. under the Public Functions section). Set it to NO to prevent # subgrouping. Alternatively, this can be done per class using the # \nosubgrouping command. # The default value is: YES. SUBGROUPING = YES # When the INLINE_GROUPED_CLASSES tag is set to YES, classes, structs and unions # are shown inside the group in which they are included (e.g. using \ingroup) # instead of on a separate page (for HTML and Man pages) or section (for LaTeX # and RTF). # # Note that this feature does not work in combination with # SEPARATE_MEMBER_PAGES. # The default value is: NO. INLINE_GROUPED_CLASSES = NO # When the INLINE_SIMPLE_STRUCTS tag is set to YES, structs, classes, and unions # with only public data fields or simple typedef fields will be shown inline in # the documentation of the scope in which they are defined (i.e. file, # namespace, or group documentation), provided this scope is documented. If set # to NO, structs, classes, and unions are shown on a separate page (for HTML and # Man pages) or section (for LaTeX and RTF). # The default value is: NO. INLINE_SIMPLE_STRUCTS = NO # When TYPEDEF_HIDES_STRUCT tag is enabled, a typedef of a struct, union, or # enum is documented as struct, union, or enum with the name of the typedef. So # typedef struct TypeS {} TypeT, will appear in the documentation as a struct # with name TypeT. When disabled the typedef will appear as a member of a file, # namespace, or class. And the struct will be named TypeS. This can typically be # useful for C code in case the coding convention dictates that all compound # types are typedef'ed and only the typedef is referenced, never the tag name. # The default value is: NO. TYPEDEF_HIDES_STRUCT = NO # The size of the symbol lookup cache can be set using LOOKUP_CACHE_SIZE. This # cache is used to resolve symbols given their name and scope. Since this can be # an expensive process and often the same symbol appears multiple times in the # code, doxygen keeps a cache of pre-resolved symbols. If the cache is too small # doxygen will become slower. If the cache is too large, memory is wasted. The # cache size is given by this formula: 2^(16+LOOKUP_CACHE_SIZE). The valid range # is 0..9, the default is 0, corresponding to a cache size of 2^16=65536 # symbols. At the end of a run doxygen will report the cache usage and suggest # the optimal cache size from a speed point of view. # Minimum value: 0, maximum value: 9, default value: 0. LOOKUP_CACHE_SIZE = 0 #--------------------------------------------------------------------------- # Build related configuration options #--------------------------------------------------------------------------- # If the EXTRACT_ALL tag is set to YES doxygen will assume all entities in # documentation are documented, even if no documentation was available. Private # class members and static file members will be hidden unless the # EXTRACT_PRIVATE respectively EXTRACT_STATIC tags are set to YES. # Note: This will also disable the warnings about undocumented members that are # normally produced when WARNINGS is set to YES. # The default value is: NO. EXTRACT_ALL = YES # If the EXTRACT_PRIVATE tag is set to YES all private members of a class will # be included in the documentation. # The default value is: NO. EXTRACT_PRIVATE = YES # If the EXTRACT_PACKAGE tag is set to YES all members with package or internal # scope will be included in the documentation. # The default value is: NO. EXTRACT_PACKAGE = YES # If the EXTRACT_STATIC tag is set to YES all static members of a file will be # included in the documentation. # The default value is: NO. EXTRACT_STATIC = YES # If the EXTRACT_LOCAL_CLASSES tag is set to YES classes (and structs) defined # locally in source files will be included in the documentation. If set to NO # only classes defined in header files are included. Does not have any effect # for Java sources. # The default value is: YES. EXTRACT_LOCAL_CLASSES = YES # This flag is only useful for Objective-C code. When set to YES local methods, # which are defined in the implementation section but not in the interface are # included in the documentation. If set to NO only methods in the interface are # included. # The default value is: NO. EXTRACT_LOCAL_METHODS = NO # If this flag is set to YES, the members of anonymous namespaces will be # extracted and appear in the documentation as a namespace called # 'anonymous_namespace{file}', where file will be replaced with the base name of # the file that contains the anonymous namespace. By default anonymous namespace # are hidden. # The default value is: NO. EXTRACT_ANON_NSPACES = YES # If the HIDE_UNDOC_MEMBERS tag is set to YES, doxygen will hide all # undocumented members inside documented classes or files. If set to NO these # members will be included in the various overviews, but no documentation # section is generated. This option has no effect if EXTRACT_ALL is enabled. # The default value is: NO. HIDE_UNDOC_MEMBERS = NO # If the HIDE_UNDOC_CLASSES tag is set to YES, doxygen will hide all # undocumented classes that are normally visible in the class hierarchy. If set # to NO these classes will be included in the various overviews. This option has # no effect if EXTRACT_ALL is enabled. # The default value is: NO. HIDE_UNDOC_CLASSES = NO # If the HIDE_FRIEND_COMPOUNDS tag is set to YES, doxygen will hide all friend # (class|struct|union) declarations. If set to NO these declarations will be # included in the documentation. # The default value is: NO. HIDE_FRIEND_COMPOUNDS = NO # If the HIDE_IN_BODY_DOCS tag is set to YES, doxygen will hide any # documentation blocks found inside the body of a function. If set to NO these # blocks will be appended to the function's detailed documentation block. # The default value is: NO. HIDE_IN_BODY_DOCS = NO # The INTERNAL_DOCS tag determines if documentation that is typed after a # \internal command is included. If the tag is set to NO then the documentation # will be excluded. Set it to YES to include the internal documentation. # The default value is: NO. INTERNAL_DOCS = NO # If the CASE_SENSE_NAMES tag is set to NO then doxygen will only generate file # names in lower-case letters. If set to YES upper-case letters are also # allowed. This is useful if you have classes or files whose names only differ # in case and if your file system supports case sensitive file names. Windows # and Mac users are advised to set this option to NO. # The default value is: system dependent. CASE_SENSE_NAMES = YES # If the HIDE_SCOPE_NAMES tag is set to NO then doxygen will show members with # their full class and namespace scopes in the documentation. If set to YES the # scope will be hidden. # The default value is: NO. HIDE_SCOPE_NAMES = NO # If the SHOW_INCLUDE_FILES tag is set to YES then doxygen will put a list of # the files that are included by a file in the documentation of that file. # The default value is: YES. SHOW_INCLUDE_FILES = YES # If the SHOW_GROUPED_MEMB_INC tag is set to YES then Doxygen will add for each # grouped member an include statement to the documentation, telling the reader # which file to include in order to use the member. # The default value is: NO. SHOW_GROUPED_MEMB_INC = NO # If the FORCE_LOCAL_INCLUDES tag is set to YES then doxygen will list include # files with double quotes in the documentation rather than with sharp brackets. # The default value is: NO. FORCE_LOCAL_INCLUDES = NO # If the INLINE_INFO tag is set to YES then a tag [inline] is inserted in the # documentation for inline members. # The default value is: YES. INLINE_INFO = YES # If the SORT_MEMBER_DOCS tag is set to YES then doxygen will sort the # (detailed) documentation of file and class members alphabetically by member # name. If set to NO the members will appear in declaration order. # The default value is: YES. SORT_MEMBER_DOCS = YES # If the SORT_BRIEF_DOCS tag is set to YES then doxygen will sort the brief # descriptions of file, namespace and class members alphabetically by member # name. If set to NO the members will appear in declaration order. Note that # this will also influence the order of the classes in the class list. # The default value is: NO. SORT_BRIEF_DOCS = NO # If the SORT_MEMBERS_CTORS_1ST tag is set to YES then doxygen will sort the # (brief and detailed) documentation of class members so that constructors and # destructors are listed first. If set to NO the constructors will appear in the # respective orders defined by SORT_BRIEF_DOCS and SORT_MEMBER_DOCS. # Note: If SORT_BRIEF_DOCS is set to NO this option is ignored for sorting brief # member documentation. # Note: If SORT_MEMBER_DOCS is set to NO this option is ignored for sorting # detailed member documentation. # The default value is: NO. SORT_MEMBERS_CTORS_1ST = NO # If the SORT_GROUP_NAMES tag is set to YES then doxygen will sort the hierarchy # of group names into alphabetical order. If set to NO the group names will # appear in their defined order. # The default value is: NO. SORT_GROUP_NAMES = NO # If the SORT_BY_SCOPE_NAME tag is set to YES, the class list will be sorted by # fully-qualified names, including namespaces. If set to NO, the class list will # be sorted only by class name, not including the namespace part. # Note: This option is not very useful if HIDE_SCOPE_NAMES is set to YES. # Note: This option applies only to the class list, not to the alphabetical # list. # The default value is: NO. SORT_BY_SCOPE_NAME = NO # If the STRICT_PROTO_MATCHING option is enabled and doxygen fails to do proper # type resolution of all parameters of a function it will reject a match between # the prototype and the implementation of a member function even if there is # only one candidate or it is obvious which candidate to choose by doing a # simple string match. By disabling STRICT_PROTO_MATCHING doxygen will still # accept a match between prototype and implementation in such cases. # The default value is: NO. STRICT_PROTO_MATCHING = NO # The GENERATE_TODOLIST tag can be used to enable ( YES) or disable ( NO) the # todo list. This list is created by putting \todo commands in the # documentation. # The default value is: YES. GENERATE_TODOLIST = YES # The GENERATE_TESTLIST tag can be used to enable ( YES) or disable ( NO) the # test list. This list is created by putting \test commands in the # documentation. # The default value is: YES. GENERATE_TESTLIST = YES # The GENERATE_BUGLIST tag can be used to enable ( YES) or disable ( NO) the bug # list. This list is created by putting \bug commands in the documentation. # The default value is: YES. GENERATE_BUGLIST = YES # The GENERATE_DEPRECATEDLIST tag can be used to enable ( YES) or disable ( NO) # the deprecated list. This list is created by putting \deprecated commands in # the documentation. # The default value is: YES. GENERATE_DEPRECATEDLIST= YES # The ENABLED_SECTIONS tag can be used to enable conditional documentation # sections, marked by \if ... \endif and \cond # ... \endcond blocks. ENABLED_SECTIONS = # The MAX_INITIALIZER_LINES tag determines the maximum number of lines that the # initial value of a variable or macro / define can have for it to appear in the # documentation. If the initializer consists of more lines than specified here # it will be hidden. Use a value of 0 to hide initializers completely. The # appearance of the value of individual variables and macros / defines can be # controlled using \showinitializer or \hideinitializer command in the # documentation regardless of this setting. # Minimum value: 0, maximum value: 10000, default value: 30. MAX_INITIALIZER_LINES = 30 # Set the SHOW_USED_FILES tag to NO to disable the list of files generated at # the bottom of the documentation of classes and structs. If set to YES the list # will mention the files that were used to generate the documentation. # The default value is: YES. SHOW_USED_FILES = YES # Set the SHOW_FILES tag to NO to disable the generation of the Files page. This # will remove the Files entry from the Quick Index and from the Folder Tree View # (if specified). # The default value is: YES. SHOW_FILES = YES # Set the SHOW_NAMESPACES tag to NO to disable the generation of the Namespaces # page. This will remove the Namespaces entry from the Quick Index and from the # Folder Tree View (if specified). # The default value is: YES. SHOW_NAMESPACES = YES # The FILE_VERSION_FILTER tag can be used to specify a program or script that # doxygen should invoke to get the current version for each file (typically from # the version control system). Doxygen will invoke the program by executing (via # popen()) the command command input-file, where command is the value of the # FILE_VERSION_FILTER tag, and input-file is the name of an input file provided # by doxygen. Whatever the program writes to standard output is used as the file # version. For an example see the documentation. FILE_VERSION_FILTER = # The LAYOUT_FILE tag can be used to specify a layout file which will be parsed # by doxygen. The layout file controls the global structure of the generated # output files in an output format independent way. To create the layout file # that represents doxygen's defaults, run doxygen with the -l option. You can # optionally specify a file name after the option, if omitted DoxygenLayout.xml # will be used as the name of the layout file. # # Note that if you run doxygen from a directory containing a file called # DoxygenLayout.xml, doxygen will parse it automatically even if the LAYOUT_FILE # tag is left empty. LAYOUT_FILE = # The CITE_BIB_FILES tag can be used to specify one or more bib files containing # the reference definitions. This must be a list of .bib files. The .bib # extension is automatically appended if omitted. This requires the bibtex tool # to be installed. See also http://en.wikipedia.org/wiki/BibTeX for more info. # For LaTeX the style of the bibliography can be controlled using # LATEX_BIB_STYLE. To use this feature you need bibtex and perl available in the # search path. Do not use file names with spaces, bibtex cannot handle them. See # also \cite for info how to create references. CITE_BIB_FILES = #--------------------------------------------------------------------------- # Configuration options related to warning and progress messages #--------------------------------------------------------------------------- # The QUIET tag can be used to turn on/off the messages that are generated to # standard output by doxygen. If QUIET is set to YES this implies that the # messages are off. # The default value is: NO. QUIET = NO # The WARNINGS tag can be used to turn on/off the warning messages that are # generated to standard error ( stderr) by doxygen. If WARNINGS is set to YES # this implies that the warnings are on. # # Tip: Turn warnings on while writing the documentation. # The default value is: YES. WARNINGS = YES # If the WARN_IF_UNDOCUMENTED tag is set to YES, then doxygen will generate # warnings for undocumented members. If EXTRACT_ALL is set to YES then this flag # will automatically be disabled. # The default value is: YES. WARN_IF_UNDOCUMENTED = YES # If the WARN_IF_DOC_ERROR tag is set to YES, doxygen will generate warnings for # potential errors in the documentation, such as not documenting some parameters # in a documented function, or documenting parameters that don't exist or using # markup commands wrongly. # The default value is: YES. WARN_IF_DOC_ERROR = YES # This WARN_NO_PARAMDOC option can be enabled to get warnings for functions that # are documented, but have no documentation for their parameters or return # value. If set to NO doxygen will only warn about wrong or incomplete parameter # documentation, but not about the absence of documentation. # The default value is: NO. WARN_NO_PARAMDOC = NO # The WARN_FORMAT tag determines the format of the warning messages that doxygen # can produce. The string should contain the $file, $line, and $text tags, which # will be replaced by the file and line number from which the warning originated # and the warning text. Optionally the format may contain $version, which will # be replaced by the version of the file (if it could be obtained via # FILE_VERSION_FILTER) # The default value is: $file:$line: $text. WARN_FORMAT = "$file:$line: $text" # The WARN_LOGFILE tag can be used to specify a file to which warning and error # messages should be written. If left blank the output is written to standard # error (stderr). WARN_LOGFILE = #--------------------------------------------------------------------------- # Configuration options related to the input files #--------------------------------------------------------------------------- # The INPUT tag is used to specify the files and/or directories that contain # documented source files. You may enter file names like myfile.cpp or # directories like /usr/src/myproject. Separate the files or directories with # spaces. # Note: If this tag is empty the current directory is searched. INPUT = ../ # This tag can be used to specify the character encoding of the source files # that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses # libiconv (or the iconv built into libc) for the transcoding. See the libiconv # documentation (see: http://www.gnu.org/software/libiconv) for the list of # possible encodings. # The default value is: UTF-8. INPUT_ENCODING = UTF-8 # If the value of the INPUT tag contains directories, you can use the # FILE_PATTERNS tag to specify one or more wildcard patterns (like *.cpp and # *.h) to filter out the source-files in the directories. If left blank the # following patterns are tested:*.c, *.cc, *.cxx, *.cpp, *.c++, *.java, *.ii, # *.ixx, *.ipp, *.i++, *.inl, *.idl, *.ddl, *.odl, *.h, *.hh, *.hxx, *.hpp, # *.h++, *.cs, *.d, *.php, *.php4, *.php5, *.phtml, *.inc, *.m, *.markdown, # *.md, *.mm, *.dox, *.py, *.f90, *.f, *.for, *.tcl, *.vhd, *.vhdl, *.ucf, # *.qsf, *.as and *.js. FILE_PATTERNS = # The RECURSIVE tag can be used to specify whether or not subdirectories should # be searched for input files as well. # The default value is: NO. RECURSIVE = YES # The EXCLUDE tag can be used to specify files and/or directories that should be # excluded from the INPUT source files. This way you can easily exclude a # subdirectory from a directory tree whose root is specified with the INPUT tag. # # Note that relative paths are relative to the directory from which doxygen is # run. EXCLUDE = # The EXCLUDE_SYMLINKS tag can be used to select whether or not files or # directories that are symbolic links (a Unix file system feature) are excluded # from the input. # The default value is: NO. EXCLUDE_SYMLINKS = NO # If the value of the INPUT tag contains directories, you can use the # EXCLUDE_PATTERNS tag to specify one or more wildcard patterns to exclude # certain files from those directories. # # Note that the wildcards are matched against the file with absolute path, so to # exclude all test directories for example use the pattern */test/* EXCLUDE_PATTERNS = # The EXCLUDE_SYMBOLS tag can be used to specify one or more symbol names # (namespaces, classes, functions, etc.) that should be excluded from the # output. The symbol name can be a fully qualified name, a word, or if the # wildcard * is used, a substring. Examples: ANamespace, AClass, # AClass::ANamespace, ANamespace::*Test # # Note that the wildcards are matched against the file with absolute path, so to # exclude all test directories use the pattern */test/* EXCLUDE_SYMBOLS = # The EXAMPLE_PATH tag can be used to specify one or more files or directories # that contain example code fragments that are included (see the \include # command). EXAMPLE_PATH = # If the value of the EXAMPLE_PATH tag contains directories, you can use the # EXAMPLE_PATTERNS tag to specify one or more wildcard pattern (like *.cpp and # *.h) to filter out the source-files in the directories. If left blank all # files are included. EXAMPLE_PATTERNS = # If the EXAMPLE_RECURSIVE tag is set to YES then subdirectories will be # searched for input files to be used with the \include or \dontinclude commands # irrespective of the value of the RECURSIVE tag. # The default value is: NO. EXAMPLE_RECURSIVE = NO # The IMAGE_PATH tag can be used to specify one or more files or directories # that contain images that are to be included in the documentation (see the # \image command). IMAGE_PATH = # The INPUT_FILTER tag can be used to specify a program that doxygen should # invoke to filter for each input file. Doxygen will invoke the filter program # by executing (via popen()) the command: # # # # where is the value of the INPUT_FILTER tag, and is the # name of an input file. Doxygen will then use the output that the filter # program writes to standard output. If FILTER_PATTERNS is specified, this tag # will be ignored. # # Note that the filter must not add or remove lines; it is applied before the # code is scanned, but not when the output code is generated. If lines are added # or removed, the anchors will not be placed correctly. INPUT_FILTER = # The FILTER_PATTERNS tag can be used to specify filters on a per file pattern # basis. Doxygen will compare the file name with each pattern and apply the # filter if there is a match. The filters are a list of the form: pattern=filter # (like *.cpp=my_cpp_filter). See INPUT_FILTER for further information on how # filters are used. If the FILTER_PATTERNS tag is empty or if none of the # patterns match the file name, INPUT_FILTER is applied. FILTER_PATTERNS = # If the FILTER_SOURCE_FILES tag is set to YES, the input filter (if set using # INPUT_FILTER ) will also be used to filter the input files that are used for # producing the source files to browse (i.e. when SOURCE_BROWSER is set to YES). # The default value is: NO. FILTER_SOURCE_FILES = NO # The FILTER_SOURCE_PATTERNS tag can be used to specify source filters per file # pattern. A pattern will override the setting for FILTER_PATTERN (if any) and # it is also possible to disable source filtering for a specific pattern using # *.ext= (so without naming a filter). # This tag requires that the tag FILTER_SOURCE_FILES is set to YES. FILTER_SOURCE_PATTERNS = # If the USE_MDFILE_AS_MAINPAGE tag refers to the name of a markdown file that # is part of the input, its contents will be placed on the main page # (index.html). This can be useful if you have a project on for instance GitHub # and want to reuse the introduction page also for the doxygen output. USE_MDFILE_AS_MAINPAGE = #--------------------------------------------------------------------------- # Configuration options related to source browsing #--------------------------------------------------------------------------- # If the SOURCE_BROWSER tag is set to YES then a list of source files will be # generated. Documented entities will be cross-referenced with these sources. # # Note: To get rid of all source code in the generated output, make sure that # also VERBATIM_HEADERS is set to NO. # The default value is: NO. SOURCE_BROWSER = YES # Setting the INLINE_SOURCES tag to YES will include the body of functions, # classes and enums directly into the documentation. # The default value is: NO. INLINE_SOURCES = YES # Setting the STRIP_CODE_COMMENTS tag to YES will instruct doxygen to hide any # special comment blocks from generated source code fragments. Normal C, C++ and # Fortran comments will always remain visible. # The default value is: YES. STRIP_CODE_COMMENTS = YES # If the REFERENCED_BY_RELATION tag is set to YES then for each documented # function all documented functions referencing it will be listed. # The default value is: NO. REFERENCED_BY_RELATION = YES # If the REFERENCES_RELATION tag is set to YES then for each documented function # all documented entities called/used by that function will be listed. # The default value is: NO. REFERENCES_RELATION = YES # If the REFERENCES_LINK_SOURCE tag is set to YES and SOURCE_BROWSER tag is set # to YES, then the hyperlinks from functions in REFERENCES_RELATION and # REFERENCED_BY_RELATION lists will link to the source code. Otherwise they will # link to the documentation. # The default value is: YES. REFERENCES_LINK_SOURCE = YES # If SOURCE_TOOLTIPS is enabled (the default) then hovering a hyperlink in the # source code will show a tooltip with additional information such as prototype, # brief description and links to the definition and documentation. Since this # will make the HTML file larger and loading of large files a bit slower, you # can opt to disable this feature. # The default value is: YES. # This tag requires that the tag SOURCE_BROWSER is set to YES. SOURCE_TOOLTIPS = YES # If the USE_HTAGS tag is set to YES then the references to source code will # point to the HTML generated by the htags(1) tool instead of doxygen built-in # source browser. The htags tool is part of GNU's global source tagging system # (see http://www.gnu.org/software/global/global.html). You will need version # 4.8.6 or higher. # # To use it do the following: # - Install the latest version of global # - Enable SOURCE_BROWSER and USE_HTAGS in the config file # - Make sure the INPUT points to the root of the source tree # - Run doxygen as normal # # Doxygen will invoke htags (and that will in turn invoke gtags), so these # tools must be available from the command line (i.e. in the search path). # # The result: instead of the source browser generated by doxygen, the links to # source code will now point to the output of htags. # The default value is: NO. # This tag requires that the tag SOURCE_BROWSER is set to YES. USE_HTAGS = NO # If the VERBATIM_HEADERS tag is set the YES then doxygen will generate a # verbatim copy of the header file for each class for which an include is # specified. Set to NO to disable this. # See also: Section \class. # The default value is: YES. VERBATIM_HEADERS = YES #--------------------------------------------------------------------------- # Configuration options related to the alphabetical class index #--------------------------------------------------------------------------- # If the ALPHABETICAL_INDEX tag is set to YES, an alphabetical index of all # compounds will be generated. Enable this if the project contains a lot of # classes, structs, unions or interfaces. # The default value is: YES. ALPHABETICAL_INDEX = YES # The COLS_IN_ALPHA_INDEX tag can be used to specify the number of columns in # which the alphabetical index list will be split. # Minimum value: 1, maximum value: 20, default value: 5. # This tag requires that the tag ALPHABETICAL_INDEX is set to YES. COLS_IN_ALPHA_INDEX = 5 # In case all classes in a project start with a common prefix, all classes will # be put under the same header in the alphabetical index. The IGNORE_PREFIX tag # can be used to specify a prefix (or a list of prefixes) that should be ignored # while generating the index headers. # This tag requires that the tag ALPHABETICAL_INDEX is set to YES. IGNORE_PREFIX = #--------------------------------------------------------------------------- # Configuration options related to the HTML output #--------------------------------------------------------------------------- # If the GENERATE_HTML tag is set to YES doxygen will generate HTML output # The default value is: YES. GENERATE_HTML = YES # The HTML_OUTPUT tag is used to specify where the HTML docs will be put. If a # relative path is entered the value of OUTPUT_DIRECTORY will be put in front of # it. # The default directory is: html. # This tag requires that the tag GENERATE_HTML is set to YES. HTML_OUTPUT = html # The HTML_FILE_EXTENSION tag can be used to specify the file extension for each # generated HTML page (for example: .htm, .php, .asp). # The default value is: .html. # This tag requires that the tag GENERATE_HTML is set to YES. HTML_FILE_EXTENSION = .html # The HTML_HEADER tag can be used to specify a user-defined HTML header file for # each generated HTML page. If the tag is left blank doxygen will generate a # standard header. # # To get valid HTML the header file that includes any scripts and style sheets # that doxygen needs, which is dependent on the configuration options used (e.g. # the setting GENERATE_TREEVIEW). It is highly recommended to start with a # default header using # doxygen -w html new_header.html new_footer.html new_stylesheet.css # YourConfigFile # and then modify the file new_header.html. See also section "Doxygen usage" # for information on how to generate the default header that doxygen normally # uses. # Note: The header is subject to change so you typically have to regenerate the # default header when upgrading to a newer version of doxygen. For a description # of the possible markers and block names see the documentation. # This tag requires that the tag GENERATE_HTML is set to YES. HTML_HEADER = # The HTML_FOOTER tag can be used to specify a user-defined HTML footer for each # generated HTML page. If the tag is left blank doxygen will generate a standard # footer. See HTML_HEADER for more information on how to generate a default # footer and what special commands can be used inside the footer. See also # section "Doxygen usage" for information on how to generate the default footer # that doxygen normally uses. # This tag requires that the tag GENERATE_HTML is set to YES. HTML_FOOTER = # The HTML_STYLESHEET tag can be used to specify a user-defined cascading style # sheet that is used by each HTML page. It can be used to fine-tune the look of # the HTML output. If left blank doxygen will generate a default style sheet. # See also section "Doxygen usage" for information on how to generate the style # sheet that doxygen normally uses. # Note: It is recommended to use HTML_EXTRA_STYLESHEET instead of this tag, as # it is more robust and this tag (HTML_STYLESHEET) will in the future become # obsolete. # This tag requires that the tag GENERATE_HTML is set to YES. HTML_STYLESHEET = # The HTML_EXTRA_STYLESHEET tag can be used to specify an additional user- # defined cascading style sheet that is included after the standard style sheets # created by doxygen. Using this option one can overrule certain style aspects. # This is preferred over using HTML_STYLESHEET since it does not replace the # standard style sheet and is therefor more robust against future updates. # Doxygen will copy the style sheet file to the output directory. For an example # see the documentation. # This tag requires that the tag GENERATE_HTML is set to YES. HTML_EXTRA_STYLESHEET = # The HTML_EXTRA_FILES tag can be used to specify one or more extra images or # other source files which should be copied to the HTML output directory. Note # that these files will be copied to the base HTML output directory. Use the # $relpath^ marker in the HTML_HEADER and/or HTML_FOOTER files to load these # files. In the HTML_STYLESHEET file, use the file name only. Also note that the # files will be copied as-is; there are no commands or markers available. # This tag requires that the tag GENERATE_HTML is set to YES. HTML_EXTRA_FILES = # The HTML_COLORSTYLE_HUE tag controls the color of the HTML output. Doxygen # will adjust the colors in the stylesheet and background images according to # this color. Hue is specified as an angle on a colorwheel, see # http://en.wikipedia.org/wiki/Hue for more information. For instance the value # 0 represents red, 60 is yellow, 120 is green, 180 is cyan, 240 is blue, 300 # purple, and 360 is red again. # Minimum value: 0, maximum value: 359, default value: 220. # This tag requires that the tag GENERATE_HTML is set to YES. HTML_COLORSTYLE_HUE = 220 # The HTML_COLORSTYLE_SAT tag controls the purity (or saturation) of the colors # in the HTML output. For a value of 0 the output will use grayscales only. A # value of 255 will produce the most vivid colors. # Minimum value: 0, maximum value: 255, default value: 100. # This tag requires that the tag GENERATE_HTML is set to YES. HTML_COLORSTYLE_SAT = 100 # The HTML_COLORSTYLE_GAMMA tag controls the gamma correction applied to the # luminance component of the colors in the HTML output. Values below 100 # gradually make the output lighter, whereas values above 100 make the output # darker. The value divided by 100 is the actual gamma applied, so 80 represents # a gamma of 0.8, The value 220 represents a gamma of 2.2, and 100 does not # change the gamma. # Minimum value: 40, maximum value: 240, default value: 80. # This tag requires that the tag GENERATE_HTML is set to YES. HTML_COLORSTYLE_GAMMA = 80 # If the HTML_TIMESTAMP tag is set to YES then the footer of each generated HTML # page will contain the date and time when the page was generated. Setting this # to NO can help when comparing the output of multiple runs. # The default value is: YES. # This tag requires that the tag GENERATE_HTML is set to YES. HTML_TIMESTAMP = YES # If the HTML_DYNAMIC_SECTIONS tag is set to YES then the generated HTML # documentation will contain sections that can be hidden and shown after the # page has loaded. # The default value is: NO. # This tag requires that the tag GENERATE_HTML is set to YES. HTML_DYNAMIC_SECTIONS = NO # With HTML_INDEX_NUM_ENTRIES one can control the preferred number of entries # shown in the various tree structured indices initially; the user can expand # and collapse entries dynamically later on. Doxygen will expand the tree to # such a level that at most the specified number of entries are visible (unless # a fully collapsed tree already exceeds this amount). So setting the number of # entries 1 will produce a full collapsed tree by default. 0 is a special value # representing an infinite number of entries and will result in a full expanded # tree by default. # Minimum value: 0, maximum value: 9999, default value: 100. # This tag requires that the tag GENERATE_HTML is set to YES. HTML_INDEX_NUM_ENTRIES = 100 # If the GENERATE_DOCSET tag is set to YES, additional index files will be # generated that can be used as input for Apple's Xcode 3 integrated development # environment (see: http://developer.apple.com/tools/xcode/), introduced with # OSX 10.5 (Leopard). To create a documentation set, doxygen will generate a # Makefile in the HTML output directory. Running make will produce the docset in # that directory and running make install will install the docset in # ~/Library/Developer/Shared/Documentation/DocSets so that Xcode will find it at # startup. See http://developer.apple.com/tools/creatingdocsetswithdoxygen.html # for more information. # The default value is: NO. # This tag requires that the tag GENERATE_HTML is set to YES. GENERATE_DOCSET = NO # This tag determines the name of the docset feed. A documentation feed provides # an umbrella under which multiple documentation sets from a single provider # (such as a company or product suite) can be grouped. # The default value is: Doxygen generated docs. # This tag requires that the tag GENERATE_DOCSET is set to YES. DOCSET_FEEDNAME = "Doxygen generated docs" # This tag specifies a string that should uniquely identify the documentation # set bundle. This should be a reverse domain-name style string, e.g. # com.mycompany.MyDocSet. Doxygen will append .docset to the name. # The default value is: org.doxygen.Project. # This tag requires that the tag GENERATE_DOCSET is set to YES. DOCSET_BUNDLE_ID = org.doxygen.Project # The DOCSET_PUBLISHER_ID tag specifies a string that should uniquely identify # the documentation publisher. This should be a reverse domain-name style # string, e.g. com.mycompany.MyDocSet.documentation. # The default value is: org.doxygen.Publisher. # This tag requires that the tag GENERATE_DOCSET is set to YES. DOCSET_PUBLISHER_ID = org.doxygen.Publisher # The DOCSET_PUBLISHER_NAME tag identifies the documentation publisher. # The default value is: Publisher. # This tag requires that the tag GENERATE_DOCSET is set to YES. DOCSET_PUBLISHER_NAME = Publisher # If the GENERATE_HTMLHELP tag is set to YES then doxygen generates three # additional HTML index files: index.hhp, index.hhc, and index.hhk. The # index.hhp is a project file that can be read by Microsoft's HTML Help Workshop # (see: http://www.microsoft.com/en-us/download/details.aspx?id=21138) on # Windows. # # The HTML Help Workshop contains a compiler that can convert all HTML output # generated by doxygen into a single compiled HTML file (.chm). Compiled HTML # files are now used as the Windows 98 help format, and will replace the old # Windows help format (.hlp) on all Windows platforms in the future. Compressed # HTML files also contain an index, a table of contents, and you can search for # words in the documentation. The HTML workshop also contains a viewer for # compressed HTML files. # The default value is: NO. # This tag requires that the tag GENERATE_HTML is set to YES. GENERATE_HTMLHELP = NO # The CHM_FILE tag can be used to specify the file name of the resulting .chm # file. You can add a path in front of the file if the result should not be # written to the html output directory. # This tag requires that the tag GENERATE_HTMLHELP is set to YES. CHM_FILE = # The HHC_LOCATION tag can be used to specify the location (absolute path # including file name) of the HTML help compiler ( hhc.exe). If non-empty # doxygen will try to run the HTML help compiler on the generated index.hhp. # The file has to be specified with full path. # This tag requires that the tag GENERATE_HTMLHELP is set to YES. HHC_LOCATION = # The GENERATE_CHI flag controls if a separate .chi index file is generated ( # YES) or that it should be included in the master .chm file ( NO). # The default value is: NO. # This tag requires that the tag GENERATE_HTMLHELP is set to YES. GENERATE_CHI = NO # The CHM_INDEX_ENCODING is used to encode HtmlHelp index ( hhk), content ( hhc) # and project file content. # This tag requires that the tag GENERATE_HTMLHELP is set to YES. CHM_INDEX_ENCODING = # The BINARY_TOC flag controls whether a binary table of contents is generated ( # YES) or a normal table of contents ( NO) in the .chm file. # The default value is: NO. # This tag requires that the tag GENERATE_HTMLHELP is set to YES. BINARY_TOC = NO # The TOC_EXPAND flag can be set to YES to add extra items for group members to # the table of contents of the HTML help documentation and to the tree view. # The default value is: NO. # This tag requires that the tag GENERATE_HTMLHELP is set to YES. TOC_EXPAND = NO # If the GENERATE_QHP tag is set to YES and both QHP_NAMESPACE and # QHP_VIRTUAL_FOLDER are set, an additional index file will be generated that # can be used as input for Qt's qhelpgenerator to generate a Qt Compressed Help # (.qch) of the generated HTML documentation. # The default value is: NO. # This tag requires that the tag GENERATE_HTML is set to YES. GENERATE_QHP = NO # If the QHG_LOCATION tag is specified, the QCH_FILE tag can be used to specify # the file name of the resulting .qch file. The path specified is relative to # the HTML output folder. # This tag requires that the tag GENERATE_QHP is set to YES. QCH_FILE = # The QHP_NAMESPACE tag specifies the namespace to use when generating Qt Help # Project output. For more information please see Qt Help Project / Namespace # (see: http://qt-project.org/doc/qt-4.8/qthelpproject.html#namespace). # The default value is: org.doxygen.Project. # This tag requires that the tag GENERATE_QHP is set to YES. QHP_NAMESPACE = org.doxygen.Project # The QHP_VIRTUAL_FOLDER tag specifies the namespace to use when generating Qt # Help Project output. For more information please see Qt Help Project / Virtual # Folders (see: http://qt-project.org/doc/qt-4.8/qthelpproject.html#virtual- # folders). # The default value is: doc. # This tag requires that the tag GENERATE_QHP is set to YES. QHP_VIRTUAL_FOLDER = doc # If the QHP_CUST_FILTER_NAME tag is set, it specifies the name of a custom # filter to add. For more information please see Qt Help Project / Custom # Filters (see: http://qt-project.org/doc/qt-4.8/qthelpproject.html#custom- # filters). # This tag requires that the tag GENERATE_QHP is set to YES. QHP_CUST_FILTER_NAME = # The QHP_CUST_FILTER_ATTRS tag specifies the list of the attributes of the # custom filter to add. For more information please see Qt Help Project / Custom # Filters (see: http://qt-project.org/doc/qt-4.8/qthelpproject.html#custom- # filters). # This tag requires that the tag GENERATE_QHP is set to YES. QHP_CUST_FILTER_ATTRS = # The QHP_SECT_FILTER_ATTRS tag specifies the list of the attributes this # project's filter section matches. Qt Help Project / Filter Attributes (see: # http://qt-project.org/doc/qt-4.8/qthelpproject.html#filter-attributes). # This tag requires that the tag GENERATE_QHP is set to YES. QHP_SECT_FILTER_ATTRS = # The QHG_LOCATION tag can be used to specify the location of Qt's # qhelpgenerator. If non-empty doxygen will try to run qhelpgenerator on the # generated .qhp file. # This tag requires that the tag GENERATE_QHP is set to YES. QHG_LOCATION = # If the GENERATE_ECLIPSEHELP tag is set to YES, additional index files will be # generated, together with the HTML files, they form an Eclipse help plugin. To # install this plugin and make it available under the help contents menu in # Eclipse, the contents of the directory containing the HTML and XML files needs # to be copied into the plugins directory of eclipse. The name of the directory # within the plugins directory should be the same as the ECLIPSE_DOC_ID value. # After copying Eclipse needs to be restarted before the help appears. # The default value is: NO. # This tag requires that the tag GENERATE_HTML is set to YES. GENERATE_ECLIPSEHELP = NO # A unique identifier for the Eclipse help plugin. When installing the plugin # the directory name containing the HTML and XML files should also have this # name. Each documentation set should have its own identifier. # The default value is: org.doxygen.Project. # This tag requires that the tag GENERATE_ECLIPSEHELP is set to YES. ECLIPSE_DOC_ID = org.doxygen.Project # If you want full control over the layout of the generated HTML pages it might # be necessary to disable the index and replace it with your own. The # DISABLE_INDEX tag can be used to turn on/off the condensed index (tabs) at top # of each HTML page. A value of NO enables the index and the value YES disables # it. Since the tabs in the index contain the same information as the navigation # tree, you can set this option to YES if you also set GENERATE_TREEVIEW to YES. # The default value is: NO. # This tag requires that the tag GENERATE_HTML is set to YES. DISABLE_INDEX = NO # The GENERATE_TREEVIEW tag is used to specify whether a tree-like index # structure should be generated to display hierarchical information. If the tag # value is set to YES, a side panel will be generated containing a tree-like # index structure (just like the one that is generated for HTML Help). For this # to work a browser that supports JavaScript, DHTML, CSS and frames is required # (i.e. any modern browser). Windows users are probably better off using the # HTML help feature. Via custom stylesheets (see HTML_EXTRA_STYLESHEET) one can # further fine-tune the look of the index. As an example, the default style # sheet generated by doxygen has an example that shows how to put an image at # the root of the tree instead of the PROJECT_NAME. Since the tree basically has # the same information as the tab index, you could consider setting # DISABLE_INDEX to YES when enabling this option. # The default value is: NO. # This tag requires that the tag GENERATE_HTML is set to YES. GENERATE_TREEVIEW = NO # The ENUM_VALUES_PER_LINE tag can be used to set the number of enum values that # doxygen will group on one line in the generated HTML documentation. # # Note that a value of 0 will completely suppress the enum values from appearing # in the overview section. # Minimum value: 0, maximum value: 20, default value: 4. # This tag requires that the tag GENERATE_HTML is set to YES. ENUM_VALUES_PER_LINE = 4 # If the treeview is enabled (see GENERATE_TREEVIEW) then this tag can be used # to set the initial width (in pixels) of the frame in which the tree is shown. # Minimum value: 0, maximum value: 1500, default value: 250. # This tag requires that the tag GENERATE_HTML is set to YES. TREEVIEW_WIDTH = 250 # When the EXT_LINKS_IN_WINDOW option is set to YES doxygen will open links to # external symbols imported via tag files in a separate window. # The default value is: NO. # This tag requires that the tag GENERATE_HTML is set to YES. EXT_LINKS_IN_WINDOW = NO # Use this tag to change the font size of LaTeX formulas included as images in # the HTML documentation. When you change the font size after a successful # doxygen run you need to manually remove any form_*.png images from the HTML # output directory to force them to be regenerated. # Minimum value: 8, maximum value: 50, default value: 10. # This tag requires that the tag GENERATE_HTML is set to YES. FORMULA_FONTSIZE = 10 # Use the FORMULA_TRANPARENT tag to determine whether or not the images # generated for formulas are transparent PNGs. Transparent PNGs are not # supported properly for IE 6.0, but are supported on all modern browsers. # # Note that when changing this option you need to delete any form_*.png files in # the HTML output directory before the changes have effect. # The default value is: YES. # This tag requires that the tag GENERATE_HTML is set to YES. FORMULA_TRANSPARENT = YES # Enable the USE_MATHJAX option to render LaTeX formulas using MathJax (see # http://www.mathjax.org) which uses client side Javascript for the rendering # instead of using prerendered bitmaps. Use this if you do not have LaTeX # installed or if you want to formulas look prettier in the HTML output. When # enabled you may also need to install MathJax separately and configure the path # to it using the MATHJAX_RELPATH option. # The default value is: NO. # This tag requires that the tag GENERATE_HTML is set to YES. USE_MATHJAX = NO # When MathJax is enabled you can set the default output format to be used for # the MathJax output. See the MathJax site (see: # http://docs.mathjax.org/en/latest/output.html) for more details. # Possible values are: HTML-CSS (which is slower, but has the best # compatibility), NativeMML (i.e. MathML) and SVG. # The default value is: HTML-CSS. # This tag requires that the tag USE_MATHJAX is set to YES. MATHJAX_FORMAT = HTML-CSS # When MathJax is enabled you need to specify the location relative to the HTML # output directory using the MATHJAX_RELPATH option. The destination directory # should contain the MathJax.js script. For instance, if the mathjax directory # is located at the same level as the HTML output directory, then # MATHJAX_RELPATH should be ../mathjax. The default value points to the MathJax # Content Delivery Network so you can quickly see the result without installing # MathJax. However, it is strongly recommended to install a local copy of # MathJax from http://www.mathjax.org before deployment. # The default value is: http://cdn.mathjax.org/mathjax/latest. # This tag requires that the tag USE_MATHJAX is set to YES. MATHJAX_RELPATH = http://cdn.mathjax.org/mathjax/latest # The MATHJAX_EXTENSIONS tag can be used to specify one or more MathJax # extension names that should be enabled during MathJax rendering. For example # MATHJAX_EXTENSIONS = TeX/AMSmath TeX/AMSsymbols # This tag requires that the tag USE_MATHJAX is set to YES. MATHJAX_EXTENSIONS = # The MATHJAX_CODEFILE tag can be used to specify a file with javascript pieces # of code that will be used on startup of the MathJax code. See the MathJax site # (see: http://docs.mathjax.org/en/latest/output.html) for more details. For an # example see the documentation. # This tag requires that the tag USE_MATHJAX is set to YES. MATHJAX_CODEFILE = # When the SEARCHENGINE tag is enabled doxygen will generate a search box for # the HTML output. The underlying search engine uses javascript and DHTML and # should work on any modern browser. Note that when using HTML help # (GENERATE_HTMLHELP), Qt help (GENERATE_QHP), or docsets (GENERATE_DOCSET) # there is already a search function so this one should typically be disabled. # For large projects the javascript based search engine can be slow, then # enabling SERVER_BASED_SEARCH may provide a better solution. It is possible to # search using the keyboard; to jump to the search box use + S # (what the is depends on the OS and browser, but it is typically # , /