.. _variogram: .. private Variograms ########## The typical variogram modeling workflow is an iterative process: 1. Examine the data to determine a reasonable set of directions and data spacing values for lag distances 2. Calculate experimental variograms in the chosen directions (major, minor, vertical) * Refine these direction choices iteratively, considering omnidirectional and alternative variogram types (PR, Correlogram etc) 3. Evaluate and interpret the calculated variograms 4. Model the calculated variograms with a consistent 3D model * Tune the variogram model with the assistance of automatic fitting 5. Use the variogram, and possible come back around to compare with simulation results Introduction ============ Variogram modeling in pygeostat centers around the :ref:`gs.Variogram `. A brief overview on python class objects can be found in the the :ref:`python introduction ` section. As a quick summary, the :ref:`gs.Variogram ` class contains: * Wrappers for variogram calculation, modeling, and plotting functions * Container for experimental variogram calculation parameters for both point and gridded data * Container for variogram modeling parameters * Container for experimental (point and gridded data) and modeled variogram data * Functions to aid in setting various variogram parameters As a very quick example, lets set-up a variogram object: >>> vario = gs.Variogram(datafl=datafl, var=var) On the user-end, to call variables stored within the an instance of the variogram class, you would replace ``self`` with ``vario``. For example, lets say the major directions azimuth, dip, and tilt were already saved in the object; to see what it is, you would call the variable containing that information by: >>> vario.major [40, 0, 0] There are various functions stored within the Variogram objects as well. For example, once the variogram parameters and directions are set, the experimental variogram calculation function can be used by calling: >>> vario.varcalc() Potentially considering many directions, and possibly different tolerances and types for a single direction makes establishing a generic class which does everything challenging. This class is setup for 2 common use cases: 1. Defined major, minor and vertical directions - used for calculation and modeling * In a 3D omnidirectional case, directional data is stored in the major 2. Generic set of specified parameters - flexible but requires more manual setup There are two iPython Notebooks within the pygeostat distribution. Please review them for example workflows. They are located in: >>> ./examples/variograms Variogram Object Attributes =========================== Variables and Data ++++++++++++++++++ All of the variables and data stored in the :class:`gs.Variogram ` class are as follows: .. tabularcolumns:: |\Y{0.2}|\Y{0.8}| =================== =============================================================================== Variables Description =================== =============================================================================== ``self.datafl`` The :class:`gs.DataFile ` class that contains the required data ``self.ndim`` The number of dimensions of the system being investigated ``self.ndir`` The number of principle directions being calculated ``self.omnihz`` Indicator if a horizontal omni-directional variogram is being used ``self.omni3d`` Indicator if a 3D omni-directional variogram is being used ``self.calcpars`` Dictionary containing variogram calculation parameters ``self.modelpars`` Dictionary containing variogram modeling parameters ``self.simpars`` Dictionary containing gridded data variogram calculation parameters ``self.directions`` List containing boolean indicators if the [Major, Minor, and Vertical] directions are being used ``self.major`` List containing the azimuth, dip, and tilt of major direction ``self.minor`` List containing the azimuth, dip, and tilt of minor direction ``self.vert`` List containing the azimuth, dip, and tilt of vertical direction ``self.majorexp`` Pandas dataframe containing experimental variogram information ``self.minorexp`` Pandas dataframe containing experimental variogram information ``self.vertexp`` Pandas dataframe containing experimental variogram information ``self.model`` :class:`gs.VariogramModel ` class containing the variogram model ``self.simdatfl`` The :class:`gs.DataFile ` class that contains the simulated data or a string pointing to the location of the GSLIB output simulation file (if it is too large to load to python) ``self.majorsim`` The rotation matrix equivalent of the grid steps used by :func:`gs.varsim() ` to calculate the Major direction ``self.minorsim`` The rotation matrix equivalent of the grid steps used by :func:`gs.varsim() ` to calculate the Minor direction ``self.vertsim`` The rotation matrix equivalent of the grid steps used by :func:`gs.varsim() ` to calculate the Vertical direction ``self.simexp`` Pandas dataframe containing the experimental variograms of all simulated realizations ``self.simavgexp`` Pandas dataframe containing the average experimental variograms of all simulated realizations =================== =============================================================================== .. _calcpars: Calcpars ++++++++ As indicated in the above table, the the dictionaries ``self.calcpars``, ``self.modelpars``, and ``self.simpars`` contain all of the parameters used by :func:`gs.varcalc() `, :func:`gs.varmodel() `, and :func:`gs.varsim() `. Please refer to the their documentation on specific details. The bellow tables provide information as to what each parameter is, how to update it, and if it is automatically populated, where it's information is sourced. They are only automatically filled if they are not explicitly set by the user. All parameters mimic the GSLIB programs they are derived from. .. tabularcolumns:: |\Y{0.2}|\Y{0.8}| ================== ================================================================================ varcalc Description ================== ================================================================================ ``'variotypes'`` A single, permiciple variogram type enumerator. Default value of ``1``. Update using :func:`self.update_calcpars() `. ``'variotails'`` A single string indicating which columns in ``self.datafl`` contains the tail variable being used. Update using :func:`self.update_calcpars() `. ``'varioheads'`` A single string indicating which columns in ``self.datafl`` contains the head variable being used. Update using :func:`self.update_calcpars() `. ``'azm'`` A list of as many azimuth values as their are directions being calculated. Update using either :func:`self.inferdirections `, :func:`self.setdirections `,or :func:`self.update_calcpars() `. ``'dip'`` A list of as many dip values as their are directions being calculated. Update using either :func:`self.inferdirections `, :func:`self.setdirections `,or :func:`self.update_calcpars() `. ``'tilt'`` A list of as many tilt values as their are directions being calculated. Update using either :func:`self.inferdirections `, :func:`self.setdirections `,or :func:`self.update_calcpars() `. ``'azmtol'`` A list of as many azimuth tolerance values as their are directions being calculated. Update using either :func:`self.settols ` or :func:`self.update_calcpars() `. ``'diptol'`` A list of as many dip tolerance values as their are directions being calculated. Update using either :func:`self.settols ` or :func:`self.update_calcpars() `. ``'nlags'`` A list of as many number of lags to calculate as there are directions being calculated. Update using either :func:`self.settols ` or :func:`self.update_calcpars() `. ``'lagdist'`` A list of as many lag distances to use as there are directions being calculated. Update using either :func:`self.settols ` or :func:`self.update_calcpars() `. ``'lagtol'`` A list of as many lag distance tolerance values as their are directions being calculated. Update using either :func:`self.settols ` or :func:`self.update_calcpars() `. ``'bandhorz'`` A list of as many horizontal bandwidth values as there are directions being calculated. Deafult value of ``1e+21``. Update using either :func:`self.settols ` or :func:`self.update_calcpars() `. ``'bandvert'`` A list of as many vertical bandwidth values as there are directions being calculated. Deafult value of ``1e+21``. Update using either :func:`self.settols ` or :func:`self.update_calcpars() `. ``'tmin'`` A single minimum trimming limit. Default value of ``-999.0``. Update using :func:`self.update_calcpars() `. ``'tmax'`` A single maximum trimming limit. Default value of ``1.0e21``. Update using :func:`self.update_calcpars() `. ``'variostd'`` A single boolean value indicating if the variograms should be standardized. Default value of ``False``. Update using :func:`self.update_calcpars() `. ``'variosills'`` A single sill value to standardize to if ``'variostd'`` is ``True``. Default value of ``None`` indicates that varcalc calculate an appropriate sill. Update using :func:`self.update_calcpars() `. ``'strict'`` Logical indicating whether to run some checks for common errors. Setting to true will slightly slow execution but could catch common user errors. Default value of ``True``. Update using :func:`self.update_calcpars() `. ``'variocutcat'`` Cutoff/categories for ``'variotypes'`` 9-12 otherwise these values will be ignored. Default value of ``None`` will be transformed to a list of zeroes. Update using :func:`self.update_calcpars() `. ================== ================================================================================ .. _modelpars: Modelpars +++++++++ .. tabularcolumns:: |\Y{0.2}|\Y{0.8}| ================= ================================================================================= varmodel Description ================= ================================================================================= ``'vargstr'`` List indicating an acceptable range that varmodel can use for the sill. Default value is ``[0.01, 1.0e21]`` if ``self.calcpars['variostd']`` is ``False``, otherwise it defaults to ``[1.0, 1.0]``. Update using :func:`self.update_modelpars() `. ``'nst'`` List indicating an acceptable number of variogram structure(s) that varmodel can use. Default value is ``1``. Update using :func:`self.update_modelpars() `. ``'c0'`` List indicating an acceptable range that varmodel can use for the nugget effect. Default value is ``[1.0e-6, sill]``. Update using :func:`self.update_modelpars() `. ``'it'`` List indicating an acceptable range of variogram structure types that varmodel can use. Default value is ``[1, 3]``. Update using :func:`self.update_modelpars() `. ``'cc'`` List indicating an acceptable range of values that varmodel can use for the covariance contribution of each structure. Default value is ``[0.1, sill]``. Update using :func:`self.update_modelpars() `. ``'ang1'`` List indicating an acceptable range of values that varmodel can use for the azimuth rotation angle. Default value is sourced from ``self.calcpars['azm']``. Update using :func:`self.update_modelpars() `. ``'ang2'`` List indicating an acceptable range of values that varmodel can use for the dip rotation angle. Default value is sourced from ``self.calcpars['dip']``. Update using :func:`self.update_modelpars() `. ``'ang3'`` List indicating an acceptable range of values that varmodel can use for the tilt rotation angle. Default value is sourced from ``self.calcpars['tilt']``. Update using :func:`self.update_modelpars() `. ``'ahmax'`` List indicating an acceptable range of values that varmodel can use for the major direction range. Default range is ``[1, 1.0e21]``. Update using :func:`self.update_modelpars() `. ``'ahmin'`` List indicating an acceptable range of values that varmodel can use for the minor direction range. Default range is ``[1, 1.0e21]``. Update using :func:`self.update_modelpars() `. ``'avert'`` List indicating an acceptable range of values that varmodel can use for the vertical direction range. Default range is ``[1, 1.0e21]``. Update using :func:`self.update_modelpars() `. ================= ================================================================================= .. _simpars: Simpars +++++++ .. tabularcolumns:: |\Y{0.2}|\Y{0.8}| ================= ================================================================================= varsim Description ================= ================================================================================= ``'lithfl'`` File with lithology information. Update using :func:`self.update_simpars() `. ``'lithcol'`` Column within ``'lithfl'`` that the lithology information is located. Update using :func:`self.update_simpars() `. ``'lithcode'`` Code indicating if lithology information is used (0=not used). Update using :func:`self.update_simpars() `. ``'datafl'`` A :class:`gs.DataFile ` class, a valid path to a data file containing simulated realizations values, or a list of file locations each with only 1 realization. Update using :func:`self.addsimdatafl() `. If a :class:`gs.DataFile ` is passed, a :class:`gs.GridDef ` must be attached to it. ``'nvar'`` Number of variables used to calculate the variogram values. Taken from ``self.calcpars`` unless explicitly set using :func:`self.update_simpars() `. ``'varcols'`` Column(s) within ``'datafl'`` that the variable(s) being used to calculate the variogram are located in. Only needed if ``'datafl'`` is not a :class:`gs.DataFile `, otherwise it is automatically set to ``'1 1'`` or ``'1 2'`` depending on what ``'nvar'`` is. Update using :func:`self.addsimdatafl() `. ``'tmin'`` Lower trimming limit. Default value of -999.0. Update using :func:`self.update_simpars() `. ``'tmax'`` Upper trimming limit. Default value of 1.0e21. Update using :func:`self.update_simpars() `. ``'outvargs'`` Output file for variograms of realizations. Defaults to a temporary file that is read into python then deleted. Update using :func:`self.update_simpars() `. ``'outavg'`` Output file for average variogram. Defaults to a temporary file that is read into python then deleted. Update using :func:`self.update_simpars() `. ``'griddef'`` :class:`gs.GridDef ` class. No default value. Will grab the :class:`gs.GridDef ` attached to a :class:`gs.DataFile ` passed into :func:`self.addsimdatafl() `. Otherwise, use :func:`self.addgriddef() `. ``'nreal'`` Number of realizations within the simulated realization file. Default value is calculated if ``'datafl'`` is a :class:`gs.DataFile ` class and ``'griddef'`` has been set. Update using :func:`self.update_simpars() `. ``'ndir'`` The number of principle directions being calculated. Default value is set to the value of ``self.ndir``. Update using :func:`self.update_simpars() `. ``'nlags'`` Number of lags to calculate. Default value is calculated from two values. The first being the maximum lag distance used for experimental variogram calculation, as determined from values in ``self.calcpars``. The second values used is based on the maximum lag distance calculated from the grid steps for each direction being calculated. Update using :func:`self.update_simpars() `. ``'dirs'`` A string containing lines of grid cell shift values for each direction. The default values are calculated using :func:`gs.get_varsimpars() ` which use what directional data exists within ``self.major``, ``self.minor``, and ``self.vert``. Update using :func:`self.update_simpars() `. ``'stdsill'`` Boolean indicator if the variograms should be standardized or not. Default value is ``False``. Update using :func:`self.update_simpars() `. ``'nvargs'`` Number of required variograms. The :class:`gs.Variogram ` class is only set up to handle a value of 1, which is what the default value is. Update using :func:`self.update_simpars() `. ``'vargpars'`` The used variable enumerators and variogram type. The default value uses ``'variotypes'`` from ``self.calcpars['variotypes']``. In the case where ``'nvar'`` is 1, the value is set to ``'1 1 variotypes'``. In the case where ``'nvar'`` is 2, the value is to ``'1 2 variotypes'``. Update using :func:`self.update_simpars() `. ``'tailcol'`` If ``'datafl'`` is a :class:`gs.DataFile ` class, ``'tailcol'`` needs to be the column name which contains the tail variable data. If ``'datafl'`` is a file location, ``'tailcol'`` needs to be a column number, referencing the tail variable data. Update using :func:`self.addsimdatafl() `. ``'headcol'`` If ``'datafl'`` is a :class:`gs.DataFile ` class, ``'headcol'`` needs to be the column name which contains the head variable data. If ``'datafl'`` is a file location, ``'headcol'`` needs to be a column number, referencing the head variable data. Update using :func:`self.addsimdatafl() `. ================= ================================================================================= .. _expvarpars: Experimental Variogram Types ============================ ================= ================================================================================= Value Experimental variogram type (from VarCalc) ================= ================================================================================= 1 traditional semivariogram 2 traditional cross semivariogram 3 covariance (-3 calculates variance (provided sill) -covariance) 4 correlogram (-4 calculates 1-correlation) 5 general relative semivariogram 6 pairwise relative semivariogram 7 semivariogram of logarithms 8 semimadogram 9 indicator semivariogram - continuous - requires a cutoff 10 indicator semivariogram - categorical - requires a category 11 traditional cross indicator semivariogram - continuous 12 traditional cross indicator semivariogram - categorical ================= ================================================================================= .. _modelvarpars: Model Variogram Types ===================== ================= ====================================== =========================================== Value Model variogram type (from VarModel) Equation ================= ====================================== =========================================== 1 Spherical :math:`\gamma(h) = c \left[ 1.5\frac{h}{a} - 0.5\left(\frac{h}{a}\right)^3 \right]` 2 Exponential :math:`\gamma(h) = c \left[ 1 - \exp \frac{-3h}{a} \right]` 3 Gaussian :math:`\gamma(h) = c \left[ 1 - \exp \left(\frac{-3h}{a}\right)^2 \right]` 4 Power Model :math:`\gamma(h) = c h^\omega` 5 Hole Effect :math:`\gamma(h) = c \left[ 1 - \cos\left( \frac{h}{a}\pi \right) \right]` ================= ====================================== =========================================== Which of the Major, Minor, and Vertical Direction Variables are Used? ===================================================================== Using the arguments passed during initialization, the required principle directions are determined. The following parameters are what are used: ``self.ndir``, ``self.ndim``, and ``self.omnihz``. Using those three parameters, the variable ``self.directions`` is set up as follows: .. code-block:: python if self.ndir == 3: self.directions = [True, True, True] elif self.ndir == 2 and self.omnihz and self.ndim == 3: self.directions = [True, False, True] elif self.ndir == 2 and self.ndim == 2: self.directions = [True, True, False] elif self.ndir == 1 and self.omnihz and self.ndim == 2: self.directions = [True, False, False] elif self.ndir == 1: self.directions = [True, False, False] The outcome ``self.directions`` also indicates which of the variables ``self.major``, ``self.minor``, and ``self.vert`` will contain information and in turn... ``self.majorexp``, ``self.majorsim``, ect. As an example of how the code would work above, in the case where a horizontal omni-directional variogram and a vertical variogram is being calculated, the directions ``self.major`` and ``self.vert`` are being used while ``self.minor`` is left to a value of None. .. _variogramclass: Variogram Class =============== .. autoclass:: pygeostat.variograms.variogram.Variogram Add a gs.GridDef ++++++++++++++++ .. automethod:: pygeostat.variograms.Variogram.addgriddef Add Simulated Data File +++++++++++++++++++++++ .. automethod:: pygeostat.variograms.Variogram.addsimdatfl Fit Variogram Model +++++++++++++++++++ .. automethod:: pygeostat.variograms.Variogram.fitmodel Interpret VarModel Parameters +++++++++++++++++++++++++++++ .. automethod:: pygeostat.variograms.Variogram.fitparsfromdict Infer Directions ++++++++++++++++ .. automethod:: pygeostat.variograms.Variogram.inferdirections Initialize from File ++++++++++++++++++++ .. automethod:: pygeostat.variograms.Variogram.init_from_file Legacy Output +++++++++++++ .. automethod:: pygeostat.variograms.Variogram.legacyout Plotting ++++++++ .. automethod:: pygeostat.variograms.Variogram.plot Print Parameters ++++++++++++++++ .. automethod:: pygeostat.variograms.Variogram.printpars Set Directions ++++++++++++++ .. automethod:: pygeostat.variograms.Variogram.setdirections Set Tolerances ++++++++++++++ .. automethod:: pygeostat.variograms.Variogram.settols Update Calculation Parameters +++++++++++++++++++++++++++++ .. automethod:: pygeostat.variograms.Variogram.update_calcpars Update Model Parameters +++++++++++++++++++++++ .. automethod:: pygeostat.variograms.Variogram.update_modelpars Update VarSim Parameters ++++++++++++++++++++++++ .. automethod:: pygeostat.variograms.Variogram.update_simpars Validate VarCalc Parameters +++++++++++++++++++++++++++ .. automethod:: pygeostat.variograms.Variogram.validcalcpars Validate Directions +++++++++++++++++++ .. automethod:: pygeostat.variograms.Variogram.validdirection Variogram Calculation Wrapper +++++++++++++++++++++++++++++ .. automethod:: pygeostat.variograms.Variogram.varcalc Variogram Grid Calculation Wrapper ++++++++++++++++++++++++++++++++++ .. automethod:: pygeostat.variograms.Variogram.varsim VariogramModel Class ==================== .. autoclass:: pygeostat.variograms.variogrammodel.VariogramModel Calculate cmax ++++++++++++++ .. automethod:: pygeostat.variograms.variogrammodel.VariogramModel.cmax Calculate Variogram Model Points ++++++++++++++++++++++++++++++++ .. automethod:: pygeostat.variograms.variogrammodel.VariogramModel.calcpoints Calculate Variogram Model Points along Principle Directions +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ .. automethod:: pygeostat.variograms.variogrammodel.VariogramModel.calcpoints_principle Copy ++++ .. automethod:: pygeostat.variograms.variogrammodel.VariogramModel.copy Get Covariance Between two Points +++++++++++++++++++++++++++++++++ .. automethod:: pygeostat.variograms.variogrammodel.VariogramModel.getcova Get Pairwise Covariance +++++++++++++++++++++++ .. automethod:: pygeostat.variograms.variogrammodel.VariogramModel.pairwisecova Get Pointwise Covariance ++++++++++++++++++++++++ .. automethod:: pygeostat.variograms.variogrammodel.VariogramModel.pointwisecova Print GSLIB-Style Model +++++++++++++++++++++++ .. automethod:: pygeostat.variograms.variogrammodel.VariogramModel.__str__ Parse GSLIB-Style Model +++++++++++++++++++++++ .. automethod:: pygeostat.variograms.variogrammodel.VariogramModel.parsestr Read Varfit Output ++++++++++++++++++ .. automethod:: pygeostat.variograms.variogrammodel.VariogramModel.readvarfit Read Varfit Model +++++++++++++++++ .. automethod:: pygeostat.variograms.variogrammodel.VariogramModel.readvarmodel Scale Variogram Model +++++++++++++++++++++ .. automethod:: pygeostat.variograms.variogrammodel.VariogramModel.scalevar Set Rotation Matrix +++++++++++++++++++ .. automethod:: pygeostat.variograms.variogrammodel.VariogramModel.setcova Write Variogram Model +++++++++++++++++++++ .. automethod:: pygeostat.variograms.variogrammodel.VariogramModel.write .. _varutils: Variogram Utility Functions =========================== Discrete Gaussian Model +++++++++++++++++++++++ .. autofunction:: pygeostat.variograms.variogram_utils.dgm Get VarSim Parameters to Match Directions +++++++++++++++++++++++++++++++++++++++++ .. autofunction:: pygeostat.variograms.variogram_utils.get_varsimpars .. autofunction:: pygeostat.variograms.variogram_utils.get_varsimpars_2d Legacy Output +++++++++++++ .. autofunction:: pygeostat.variograms.variogram_utils.legacyout Legacy Input ++++++++++++ .. autofunction:: pygeostat.variograms.variogram_utils.legacyvarginput Parse varfit_lmc Output File +++++++++++++++++++++++++++++++ .. autofunction:: pygeostat.variograms.variogram_utils.parseslmcvar Read Variogram Output +++++++++++++++++++++ .. autofunction:: pygeostat.variograms.variogram_utils.readvarg VarCalc Wrapper +++++++++++++++ .. autofunction:: pygeostat.variograms.variogram_utils.varcalc VarSim Wrapper ++++++++++++++ .. autofunction:: pygeostat.variograms.variogram_utils.varsim Calculate VarSim Lag Distance +++++++++++++++++++++++++++++ .. autofunction:: pygeostat.variograms.variogram_utils.varsimlagdist Varvisual Wrapper +++++++++++++++++ .. autofunction:: pygeostat.variograms.variogram_utils.varvisual