ext: Update pybind11 to version 2.6.2.

This should help reduce warning spew when building with newer compilers.
The pybind11::module type has been renamed pybind11::module_ to avoid
conflicts with c++20 modules, according to the pybind11 changelog, so
this CL also updates gem5 source to use the new type. There is
supposedly an alias pybind11::module which is for compatibility, but we
still get linker errors without changing to pybind11::module_.

Change-Id: I0acb36215b33e3a713866baec43f5af630c356ee
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/40255
Maintainer: Bobby R. Bruce <bbruce@ucdavis.edu>
Reviewed-by: Bobby R. Bruce <bbruce@ucdavis.edu>
Tested-by: kokoro <noreply+kokoro@google.com>

ext/pybind11/docs/advanced/pycpp/numpy.rst

@@ -57,17 +57,17 @@ specification.
 
     struct buffer_info {
         void *ptr;
-        ssize_t itemsize;
+        py::ssize_t itemsize;
         std::string format;
-        ssize_t ndim;
-        std::vector<ssize_t> shape;
-        std::vector<ssize_t> strides;
+        py::ssize_t ndim;
+        std::vector<py::ssize_t> shape;
+        std::vector<py::ssize_t> strides;
     };
 
 To create a C++ function that can take a Python buffer object as an argument,
 simply use the type ``py::buffer`` as one of its arguments. Buffers can exist
 in a great variety of configurations, hence some safety checks are usually
-necessary in the function body. Below, you can see an basic example on how to
+necessary in the function body. Below, you can see a basic example on how to
 define a custom constructor for the Eigen double precision matrix
 (``Eigen::MatrixXd``) type, which supports initialization from compatible
 buffer objects (e.g. a NumPy matrix).
@@ -81,7 +81,7 @@ buffer objects (e.g. a NumPy matrix).
     constexpr bool rowMajor = Matrix::Flags & Eigen::RowMajorBit;
 
     py::class_<Matrix>(m, "Matrix", py::buffer_protocol())
-        .def("__init__", [](Matrix &m, py::buffer b) {
+        .def(py::init([](py::buffer b) {
             typedef Eigen::Stride<Eigen::Dynamic, Eigen::Dynamic> Strides;
 
             /* Request a buffer descriptor from Python */
@@ -101,8 +101,8 @@ buffer objects (e.g. a NumPy matrix).
             auto map = Eigen::Map<Matrix, 0, Strides>(
                 static_cast<Scalar *>(info.ptr), info.shape[0], info.shape[1], strides);
 
-            new (&m) Matrix(map);
-        });
+            return Matrix(map);
+        }));
 
 For reference, the ``def_buffer()`` call for this Eigen data type should look
 as follows:
@@ -150,8 +150,10 @@ NumPy array containing double precision values.
 
 When it is invoked with a different type (e.g. an integer or a list of
 integers), the binding code will attempt to cast the input into a NumPy array
-of the requested type. Note that this feature requires the
-:file:`pybind11/numpy.h` header to be included.
+of the requested type. This feature requires the :file:`pybind11/numpy.h`
+header to be included. Note that :file:`pybind11/numpy.h` does not depend on
+the NumPy headers, and thus can be used without declaring a build-time
+dependency on NumPy; NumPy>=1.7.0 is a runtime dependency.
 
 Data in NumPy arrays is not guaranteed to packed in a dense manner;
 furthermore, entries can be separated by arbitrary column and row strides.
@@ -274,9 +276,9 @@ simply using ``vectorize``).
 
         py::buffer_info buf3 = result.request();
 
-        double *ptr1 = (double *) buf1.ptr,
-               *ptr2 = (double *) buf2.ptr,
-               *ptr3 = (double *) buf3.ptr;
+        double *ptr1 = static_cast<double *>(buf1.ptr);
+        double *ptr2 = static_cast<double *>(buf2.ptr);
+        double *ptr3 = static_cast<double *>(buf3.ptr);
 
         for (size_t idx = 0; idx < buf1.shape[0]; idx++)
             ptr3[idx] = ptr1[idx] + ptr2[idx];
@@ -309,17 +311,17 @@ where ``N`` gives the required dimensionality of the array:
     m.def("sum_3d", [](py::array_t<double> x) {
         auto r = x.unchecked<3>(); // x must have ndim = 3; can be non-writeable
         double sum = 0;
-        for (ssize_t i = 0; i < r.shape(0); i++)
-            for (ssize_t j = 0; j < r.shape(1); j++)
-                for (ssize_t k = 0; k < r.shape(2); k++)
+        for (py::ssize_t i = 0; i < r.shape(0); i++)
+            for (py::ssize_t j = 0; j < r.shape(1); j++)
+                for (py::ssize_t k = 0; k < r.shape(2); k++)
                     sum += r(i, j, k);
         return sum;
     });
     m.def("increment_3d", [](py::array_t<double> x) {
         auto r = x.mutable_unchecked<3>(); // Will throw if ndim != 3 or flags.writeable is false
-        for (ssize_t i = 0; i < r.shape(0); i++)
-            for (ssize_t j = 0; j < r.shape(1); j++)
-                for (ssize_t k = 0; k < r.shape(2); k++)
+        for (py::ssize_t i = 0; i < r.shape(0); i++)
+            for (py::ssize_t j = 0; j < r.shape(1); j++)
+                for (py::ssize_t k = 0; k < r.shape(2); k++)
                     r(i, j, k) += 1.0;
     }, py::arg().noconvert());
 
@@ -371,6 +373,8 @@ Ellipsis
 Python 3 provides a convenient ``...`` ellipsis notation that is often used to
 slice multidimensional arrays. For instance, the following snippet extracts the
 middle dimensions of a tensor with the first and last index set to zero.
+In Python 2, the syntactic sugar ``...`` is not available, but the singleton
+``Ellipsis`` (of type ``ellipsis``) can still be used directly.
 
 .. code-block:: python
 
@@ -384,3 +388,51 @@ operation on the C++ side:
 
    py::array a = /* A NumPy array */;
    py::array b = a[py::make_tuple(0, py::ellipsis(), 0)];
+
+.. versionchanged:: 2.6
+   ``py::ellipsis()`` is now also avaliable in Python 2.
+
+Memory view
+===========
+
+For a case when we simply want to provide a direct accessor to C/C++ buffer
+without a concrete class object, we can return a ``memoryview`` object. Suppose
+we wish to expose a ``memoryview`` for 2x4 uint8_t array, we can do the
+following:
+
+.. code-block:: cpp
+
+    const uint8_t buffer[] = {
+        0, 1, 2, 3,
+        4, 5, 6, 7
+    };
+    m.def("get_memoryview2d", []() {
+        return py::memoryview::from_buffer(
+            buffer,                                    // buffer pointer
+            { 2, 4 },                                  // shape (rows, cols)
+            { sizeof(uint8_t) * 4, sizeof(uint8_t) }   // strides in bytes
+        );
+    })
+
+This approach is meant for providing a ``memoryview`` for a C/C++ buffer not
+managed by Python. The user is responsible for managing the lifetime of the
+buffer. Using a ``memoryview`` created in this way after deleting the buffer in
+C++ side results in undefined behavior.
+
+We can also use ``memoryview::from_memory`` for a simple 1D contiguous buffer:
+
+.. code-block:: cpp
+
+    m.def("get_memoryview1d", []() {
+        return py::memoryview::from_memory(
+            buffer,               // buffer pointer
+            sizeof(uint8_t) * 8   // buffer size
+        );
+    })
+
+.. note::
+
+    ``memoryview::from_memory`` is not available in Python 2.
+
+.. versionchanged:: 2.6
+    ``memoryview::from_memory`` added.

ext/pybind11/docs/advanced/pycpp/object.rst

@@ -1,6 +1,8 @@
 Python types
 ############
 
+.. _wrappers:
+
 Available wrappers
 ==================
 
@@ -13,6 +15,13 @@ Available types include :class:`handle`, :class:`object`, :class:`bool_`,
 :class:`iterable`, :class:`iterator`, :class:`function`, :class:`buffer`,
 :class:`array`, and :class:`array_t`.
 
+.. warning::
+
+    Be sure to review the :ref:`pytypes_gotchas` before using this heavily in
+    your C++ API.
+
+.. _casting_back_and_forth:
+
 Casting back and forth
 ======================
 
@@ -47,20 +56,21 @@ This example obtains a reference to the Python ``Decimal`` class.
 .. code-block:: cpp
 
     // Equivalent to "from decimal import Decimal"
-    py::object Decimal = py::module::import("decimal").attr("Decimal");
+    py::object Decimal = py::module_::import("decimal").attr("Decimal");
 
 .. code-block:: cpp
 
     // Try to import scipy
-    py::object scipy = py::module::import("scipy");
+    py::object scipy = py::module_::import("scipy");
     return scipy.attr("__version__");
 
+
 .. _calling_python_functions:
 
 Calling Python functions
 ========================
 
-It is also possible to call Python classes, functions and methods 
+It is also possible to call Python classes, functions and methods
 via ``operator()``.
 
 .. code-block:: cpp
@@ -71,11 +81,11 @@ via ``operator()``.
 .. code-block:: cpp
 
     // Use Python to make our directories
-    py::object os = py::module::import("os");
+    py::object os = py::module_::import("os");
     py::object makedirs = os.attr("makedirs");
     makedirs("/tmp/path/to/somewhere");
 
-One can convert the result obtained from Python to a pure C++ version 
+One can convert the result obtained from Python to a pure C++ version
 if a ``py::class_`` or type conversion is defined.
 
 .. code-block:: cpp
@@ -99,8 +109,8 @@ Python method.
     py::print(py::str(exp_pi));
 
 In the example above ``pi.attr("exp")`` is a *bound method*: it will always call
-the method for that same instance of the class. Alternately one can create an 
-*unbound method* via the Python class (instead of instance) and pass the ``self`` 
+the method for that same instance of the class. Alternately one can create an
+*unbound method* via the Python class (instead of instance) and pass the ``self``
 object explicitly, followed by other arguments.
 
 .. code-block:: cpp
@@ -168,3 +178,74 @@ Generalized unpacking according to PEP448_ is also supported:
     Python functions from C++, including keywords arguments and unpacking.
 
 .. _PEP448: https://www.python.org/dev/peps/pep-0448/
+
+.. _implicit_casting:
+
+Implicit casting
+================
+
+When using the C++ interface for Python types, or calling Python functions,
+objects of type :class:`object` are returned. It is possible to invoke implicit
+conversions to subclasses like :class:`dict`. The same holds for the proxy objects
+returned by ``operator[]`` or ``obj.attr()``.
+Casting to subtypes improves code readability and allows values to be passed to
+C++ functions that require a specific subtype rather than a generic :class:`object`.
+
+.. code-block:: cpp
+
+    #include <pybind11/numpy.h>
+    using namespace pybind11::literals;
+
+    py::module_ os = py::module_::import("os");
+    py::module_ path = py::module_::import("os.path");  // like 'import os.path as path'
+    py::module_ np = py::module_::import("numpy");  // like 'import numpy as np'
+
+    py::str curdir_abs = path.attr("abspath")(path.attr("curdir"));
+    py::print(py::str("Current directory: ") + curdir_abs);
+    py::dict environ = os.attr("environ");
+    py::print(environ["HOME"]);
+    py::array_t<float> arr = np.attr("ones")(3, "dtype"_a="float32");
+    py::print(py::repr(arr + py::int_(1)));
+
+These implicit conversions are available for subclasses of :class:`object`; there
+is no need to call ``obj.cast()`` explicitly as for custom classes, see
+:ref:`casting_back_and_forth`.
+
+.. note::
+    If a trivial conversion via move constructor is not possible, both implicit and
+    explicit casting (calling ``obj.cast()``) will attempt a "rich" conversion.
+    For instance, ``py::list env = os.attr("environ");`` will succeed and is
+    equivalent to the Python code ``env = list(os.environ)`` that produces a
+    list of the dict keys.
+
+..  TODO: Adapt text once PR #2349 has landed
+
+Handling exceptions
+===================
+
+Python exceptions from wrapper classes will be thrown as a ``py::error_already_set``.
+See :ref:`Handling exceptions from Python in C++
+<handling_python_exceptions_cpp>` for more information on handling exceptions
+raised when calling C++ wrapper classes.
+
+.. _pytypes_gotchas:
+
+Gotchas
+=======
+
+Default-Constructed Wrappers
+----------------------------
+
+When a wrapper type is default-constructed, it is **not** a valid Python object (i.e. it is not ``py::none()``). It is simply the same as
+``PyObject*`` null pointer. To check for this, use
+``static_cast<bool>(my_wrapper)``.
+
+Assigning py::none() to wrappers
+--------------------------------
+
+You may be tempted to use types like ``py::str`` and ``py::dict`` in C++
+signatures (either pure C++, or in bound signatures), and assign them default
+values of ``py::none()``. However, in a best case scenario, it will fail fast
+because ``None`` is not convertible to that type (e.g. ``py::dict``), or in a
+worse case scenario, it will silently work but corrupt the types you want to
+work with (e.g. ``py::str(py::none())`` will yield ``"None"`` in Python).

ext/pybind11/docs/advanced/pycpp/utilities.rst

@@ -42,7 +42,7 @@ redirects output to the corresponding Python streams:
     m.def("noisy_func", []() {
         py::scoped_ostream_redirect stream(
             std::cout,                               // std::ostream&
-            py::module::import("sys").attr("stdout") // Python output
+            py::module_::import("sys").attr("stdout") // Python output
         );
         call_noisy_func();
     });
@@ -104,7 +104,7 @@ can be used.
     ...
 
     // Evaluate in scope of main module
-    py::object scope = py::module::import("__main__").attr("__dict__");
+    py::object scope = py::module_::import("__main__").attr("__dict__");
 
     // Evaluate an isolated expression
     int result = py::eval("my_variable + 10", scope).cast<int>();