boostorg
Add support for std::optional
It would be nice if Boost.Spirit recognized std::optional in C++17 the way it does boost::optional. In particular, optional parsers should support std::optional attributes.
Fair enough. All of this should not be so hard to support, I will look into this after completing more urgent tasks.
In my opinion std::optional and std::variant work out-of-the-box with x3.
Sample code builds without problems (gcc (Ubuntu 7.2.0-8ubuntu3) 7.2.0):
#include <boost/spirit/home/x3.hpp>
#include <variant>
#include <optional>
using optional_t = std::optional<std::string>;
using variant_t = std::variant<std::string, int>;
namespace x3 = boost::spirit::x3;
x3::rule<struct string, std::string> const string;
x3::rule<struct optional, optional_t> const optional;
x3::rule<struct variant, variant_t> const variant;
auto const string_def = *x3::alpha;
auto const optional_def = -string;
auto const variant_def = string | x3::int_;
BOOST_SPIRIT_DEFINE(string, optional, variant);
int main() {
std::string data;
optional_t o;
x3::parse(data.begin(), data.end(), optional, o);
variant_t v;
x3::parse(data.begin(), data.end(), variant, v);
return 0;
}
And std::tuple works out-of-the-box with x3 too.
Sample code builds without problems (gcc (Ubuntu 7.2.0-8ubuntu3) 7.2.0):
#include <boost/spirit/home/x3.hpp>
#include <boost/fusion/include/std_tuple.hpp>
#include <optional>
#include <variant>
#include <tuple>
using optional_t = std::optional<std::string>;
using variant_t = std::variant<std::string, int>;
using tuple_t = std::tuple<std::string, int, double>;
namespace x3 = boost::spirit::x3;
x3::rule<struct string, std::string> const string;
x3::rule<struct optional, optional_t> const optional;
x3::rule<struct variant, variant_t> const variant;
x3::rule<struct tuple, tuple_t> const tuple;
auto const string_def = *x3::alpha;
auto const optional_def = -string;
auto const variant_def = string | x3::int_;
auto const tuple_def = string >> x3::int_ >> x3::double_;
BOOST_SPIRIT_DEFINE(string, optional, variant, tuple);
int main() {
std::string data;
optional_t o;
x3::parse(data.begin(), data.end(), optional, o);
variant_t v;
x3::parse(data.begin(), data.end(), variant, v);
tuple_t t;
x3::parse(data.begin(), data.end(), tuple, t);
return 0;
}
My request was about Boost.Spirit 2.x (i.e. Boost.Spirit.Qi and possibly Karma).
I'm not familiar with x3, but will it work without BOOST_SPIRIT_DEFINE and rules? E.g. if I want to use the parser inline like this:
std::optional<int> o;
parse(data.begin(), data.end(), -int_, o);
Sample code for x3 and qi compiles without problems (gcc (Ubuntu 7.2.0-8ubuntu3) 7.2.0):
#include <boost/spirit/home/x3.hpp>
#include <boost/fusion/include/std_tuple.hpp>
#include <optional>
#include <variant>
#include <tuple>
using optional_t = std::optional<int>;
using variant_t = std::variant<int, double>;
using tuple_t = std::tuple<int, double>;
namespace x3 = boost::spirit::x3;
int main() {
std::string data;
optional_t o;
x3::parse(data.begin(), data.end(), -x3::int_, o);
variant_t v;
x3::parse(data.begin(), data.end(), x3::int_ | x3::double_, v);
tuple_t t;
x3::parse(data.begin(), data.end(), x3::int_ >> x3::double_, t);
return 0;
}
#include <boost/spirit/include/qi.hpp>
#include <boost/fusion/include/std_tuple.hpp>
#include <optional>
#include <variant>
#include <tuple>
using optional_t = std::optional<int>;
using variant_t = std::variant<int, double>;
using tuple_t = std::tuple<int, double>;
namespace qi = boost::spirit::qi;
int main() {
std::string data;
optional_t o;
qi::parse(data.begin(), data.end(), -qi::int_, o);
variant_t v;
qi::parse(data.begin(), data.end(), qi::int_ | qi::double_, v);
tuple_t t;
qi::parse(data.begin(), data.end(), qi::int_ >> qi::double_, t);
return 0;
}
This one doesn't compile for me:
#include <boost/spirit/include/qi.hpp>
#include <boost/fusion/include/std_tuple.hpp>
#include <boost/fusion/include/define_struct.hpp>
#include <optional>
#include <variant>
using optional_t = std::optional<int>;
using variant_t = std::variant<int, double>;
BOOST_FUSION_DEFINE_STRUCT
(
(my_ns), my_struct,
(optional_t, opt)
(variant_t, var)
)
namespace qi = boost::spirit::qi;
int main() {
std::string data;
std::optional<my_ns::my_struct> attr;
qi::parse(data.c_str(), data.c_str() + data.size(), -(-qi::int_ >> (qi::int_ | qi::double_)), attr);
return 0;
}
Not sure how this can be a Boost.Fusion issue - the code does compile if you change attr to my_ns::my_struct and remove the outer optional from the Spirit expression. But I'm not familiar with internals, I may be missing something.
Yes, it was just a guess because of this error message:
/home/mike/Downloads/boost_1_66_0_b1/boost_1_66_0/boost/fusion/sequence/intrinsic/begin.hpp:92:5: error: no type named ‘type’ in ‘struct boost::lazy_enable_if<boost::fusion::traits::is_sequence<int>, boost::fusion::result_of::begin<const int> >’
For what it's worth, the following hack makes boost::spirit work with std::variant:
#include <variant>
#include <boost/spirit/home/karma/operator/alternative.hpp>
#include <boost/spirit/home/qi/operator/alternative.hpp>
#include <boost/spirit/home/support/attributes.hpp>
namespace boost {
namespace spirit {
namespace traits {
template <typename... Ts, typename Domain>
struct not_is_variant<std::variant<Ts...>, Domain> : boost::mpl::false_
{
};
template <typename... Ts, typename Expected>
struct compute_compatible_component_variant<std::variant<Ts...>, Expected, mpl::false_>
{
using types = boost::mpl::vector<Ts...>;
using end = typename mpl::end<types>::type;
using iter = typename mpl::find_if<types, is_same<Expected, mpl::_1>>::type;
using distance = typename mpl::distance<typename mpl::begin<types>::type, iter>::type;
using type = typename mpl::not_<is_same<iter, end>>::type;
enum { value = type::value };
using compatible_type = typename mpl::eval_if<type, mpl::deref<iter>, mpl::identity<unused_type>>::type;
constexpr static bool is_compatible(int which) { return which == distance::value; }
};
template <typename... Ts>
struct variant_which<std::variant<Ts...>>
{
static int call(std::variant<Ts...> const & v) { return static_cast<int>(v.index()); }
};
struct VariantAttributeSizeVisitior
{
template <typename T>
std::size_t operator()(T const & val) const
{
return spirit::traits::size(val);
}
};
template <typename... Ts>
struct attribute_size<std::variant<Ts...>>
{
using type = std::size_t;
static type call(std::variant<Ts...> const & val) { return std::visit(VariantAttributeSizeVisitior(), val); }
};
} // namespace traits
namespace qi {
namespace detail {
template <typename Expected, typename... Ts>
struct find_substitute<std::variant<Ts...>, Expected>
{
using types = mpl::vector<Ts...>;
using end = typename mpl::end<types>::type;
using iter_1 = typename mpl::find_if<types, is_same<mpl::_1, Expected>>::type;
using iter = typename mpl::eval_if<is_same<iter_1, end>, mpl::find_if<types, traits::is_substitute<mpl::_1, Expected>>, mpl::identity<iter_1>>::type;
using type = typename mpl::eval_if<is_same<iter, end>, mpl::identity<Expected>, mpl::deref<iter>>::type;
};
} // namespace detail
} // namespace qi
namespace karma {
namespace detail {
template <typename Component, typename Expected, typename... Ts>
struct alternative_generate<Component,
std::variant<Ts...>,
Expected,
typename enable_if<traits::compute_compatible_component<Expected, std::variant<Ts...>, karma::domain>>::type>
{
using Attribute = std::variant<Ts...>;
template <typename OutputIterator, typename Context, typename Delimiter>
static bool call(Component const & component, OutputIterator & sink, Context & ctx, Delimiter const & d, Attribute const & attr, bool &)
{
return call(component, sink, ctx, d, attr, spirit::traits::not_is_variant<Attribute, karma::domain>());
}
template <typename OutputIterator, typename Context, typename Delimiter>
static bool call(Component const & component, OutputIterator & sink, Context & ctx, Delimiter const & d, Attribute const & attr, mpl::true_)
{
return component.generate(sink, ctx, d, attr);
}
template <typename OutputIterator, typename Context, typename Delimiter>
static bool call(Component const & component, OutputIterator & sink, Context & ctx, Delimiter const & d, Attribute const & attr, mpl::false_)
{
using component_type = traits::compute_compatible_component<Expected, Attribute, domain>;
if (!traits::has_optional_value(attr))
return false;
typename traits::optional_attribute<Attribute>::type attr_ = traits::optional_value(attr);
if (!component_type::is_compatible(spirit::traits::which(attr_)))
return false;
typedef typename component_type::compatible_type compatible_type;
return component.generate(sink, ctx, d, std::get<compatible_type>(attr_));
}
};
} // namespace detail
} // namespace karma
} // namespace spirit
} // namespace boost
It also shows that adding support for std::variant is very much straight forward. The same is probably true for std::optional.
My (little) experience with x3 and std::variant is that it works for some types, but not others (i.e., compile error). I also thought at first that it worked smoothly out of the box.
// 1) Works
std::variant<double,bool> v;
x3::parse (it, input.end(), x3::double_ | x3::bool_, v);
// 2) Compile error¹
std::variant<double,std::string> v;
x3::parse (it, input.end(), x3::double_ | +x3::alpha, v);
But if I replace std::variant with boost::variant, then the second example compiles just fine.
Maybe the problem arise when the variant contains a "container"-type? I tried to replace std::string with std::vector<char> with the same result.
It can be worked around by using rules that explicit set the attribute, like in octopus-prime's answer above.
¹ boost/spirit/home/x3/core/detail/parse_into_container.hpp:95:13: error: no type named 'type' in 'struct boost::spirit::x3::traits::container_value<std::variant<double, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, void>' value_type;
I got inspired by @fiesh's answer, and tried to do the same for Spirit X3. See the code bellow.
Now, this code works:
std::variant<double,std::string> v;
x3::parse (it, input.end(), x3::double_ | +x3::alpha, v);
But I have done very little testing, and I do not claim to fully understand the code bellow :)
Hack to make std::variant work with Spirit X3:
#include <variant>
#include <boost/spirit/home/x3/support/traits/variant_find_substitute.hpp>
// Based on: boost/spirit/home/x3/support/traits/variant_find_substitute.hpp
namespace boost::spirit::x3::traits
{
template <typename... Ts>
struct is_variant< std::variant<Ts...> > : mpl::true_ {};
template <typename... Ts, typename Attribute>
struct variant_find_substitute<std::variant<Ts...>, Attribute>
{
typedef std::variant<Ts...> variant_type;
typedef mpl::vector<Ts...> types;
typedef typename mpl::end<types>::type end;
typedef typename
mpl::find_if<types, is_same<mpl::_1, Attribute> >::type
iter_1;
typedef typename
mpl::eval_if<
is_same<iter_1, end>,
mpl::find_if<types, traits::is_substitute<mpl::_1, Attribute> >,
mpl::identity<iter_1>
>::type
iter;
typedef typename
mpl::eval_if<
is_same<iter, end>,
mpl::identity<Attribute>,
mpl::deref<iter>
>::type
type;
};
}
In case other people are also interested in getting X3 to work with std::variant, on Boost version 1.68.0, I find that torbjo's hack/patch for X3 almost worked but not quite; it's missing specialization of the variant_has_substitute_impl struct. To enable std::variant support, include the following header file in your struct declaration:
#include <variant>
#include <boost/mpl/vector.hpp>
#include <boost/spirit/home/x3/support/traits/is_variant.hpp>
#include <boost/spirit/home/x3/support/traits/tuple_traits.hpp>
#include <boost/spirit/home/x3/support/traits/variant_find_substitute.hpp>
#include <boost/spirit/home/x3/support/traits/variant_has_substitute.hpp>
// Based on: boost/spirit/home/x3/support/traits/variant_find_substitute.hpp
namespace boost::spirit::x3::traits {
template <typename... Ts> struct is_variant<std::variant<Ts...>> : mpl::true_ {};
template <typename... Ts, typename Attribute>
struct variant_find_substitute<std::variant<Ts...>, Attribute> {
typedef std::variant<Ts...> variant_type;
typedef mpl::vector<Ts...> types;
typedef typename mpl::end<types>::type end;
typedef typename mpl::find_if<types, is_same<mpl::_1, Attribute>>::type iter_1;
typedef typename mpl::eval_if<is_same<iter_1, end>,
mpl::find_if<types, traits::is_substitute<mpl::_1, Attribute>>,
mpl::identity<iter_1>>::type iter;
typedef
typename mpl::eval_if<is_same<iter, end>, mpl::identity<Attribute>, mpl::deref<iter>>::type
type;
};
template <typename... Ts, typename Attribute>
struct variant_has_substitute_impl<std::variant<Ts...>, Attribute> {
// Find a type from the variant that can be a substitute for Attribute.
// return true_ if one is found, else false_
typedef mpl::vector<Ts...> types;
typedef typename mpl::end<types>::type end;
typedef typename mpl::find_if<types, is_same<mpl::_1, Attribute>>::type iter_1;
typedef typename mpl::eval_if<is_same<iter_1, end>,
mpl::find_if<types, traits::is_substitute<mpl::_1, Attribute>>,
mpl::identity<iter_1>>::type iter;
typedef mpl::not_<is_same<iter, end>> type;
};
} // namespace boost::spirit::x3::traits
What seems to work for me is adding an identity specialization to what @pkerichang proposed (I also spiced his version up a bit):
#include <variant>
#include <boost/mpl/vector.hpp>
#include <boost/spirit/home/x3/support/traits/is_variant.hpp>
#include <boost/spirit/home/x3/support/traits/tuple_traits.hpp>
#include <boost/spirit/home/x3/support/traits/variant_find_substitute.hpp>
#include <boost/spirit/home/x3/support/traits/variant_has_substitute.hpp>
// Based on: boost/spirit/home/x3/support/traits/variant_find_substitute.hpp
namespace boost::spirit::x3::traits
{
template<typename... Ts> struct is_variant<std::variant<Ts...>> : mpl::true_ {};
template<typename... Ts, typename Attribute>
struct variant_find_substitute<std::variant<Ts...>, Attribute>
{
using variant_type = std::variant<Ts...>;
using types = mpl::vector<Ts...>;
using end = typename mpl::end<types>::type;
using iter_1 = typename mpl::find_if<types, is_same<mpl::_1, Attribute>>::type;
using iter = typename mpl::eval_if<is_same<iter_1, end>,
mpl::find_if<types, traits::is_substitute<mpl::_1, Attribute>>,
mpl::identity<iter_1>>::type;
using type = typename mpl::eval_if<is_same<iter, end>,
mpl::identity<Attribute>,
mpl::deref<iter>>::type;
};
template<typename... Ts>
struct variant_find_substitute<std::variant<Ts...>, std::variant<Ts...>>
: mpl::identity<std::variant<Ts...>> {};
template<typename... Ts, typename Attribute>
struct variant_has_substitute_impl<std::variant<Ts...>, Attribute>
{
// Find a type from the variant that can be a substitute for Attribute.
// return true_ if one is found, else false_
using types = mpl::vector<Ts...>;
using end = typename mpl::end<types>::type;
using iter_1 = typename mpl::find_if<types, is_same<mpl::_1, Attribute>>::type;
using iter = typename mpl::eval_if<is_same<iter_1, end>,
mpl::find_if<types, traits::is_substitute<mpl::_1, Attribute>>,
mpl::identity<iter_1>>::type;
using type = mpl::not_<is_same<iter, end>>;
};
}
How do I get this merged in? Where should it go in X3?
I understand that people who use C++17 want to use optional and variant from the standard library with Spirit. There are problems with the code above: 1) it requires to include definitions what is is unacceptable because including stuff from both std and boost will make Spirit slower for everyone. 2) none of them complete (X3 solution lacks specialization for save_to_assoc_attr, Qi one also lacks some) 3) The things can be simplified, but it is not a big deal.
I spend quite bunch of time to find a good general solution because without it we have to double the test suite to test both boost and std types (while it already takes an hour for a single MSVC toolset). Current variant handling in X3 also done in C++03 style and it complicates the things. Boost.Variant hurts me a lot, as well as MPL and Fusion. It is not as simple as it looks from the first sight, but the things will sort out.
The dependency problem can be solved if support for Boost or std components is enabled by including a support header. This way each user will pick what they want to use.
I understand your concerns, but
- I still don't know where to put this code inside spirit, even if incomplete (i.e. as a pull request).
- Once submitted for review, the incompleteness can be addressed properly in the right place.
- The fact that MSVC's test runs take atrociously long has little to do with adding std::variant/optional support to X3 or Qi. If it concerns you that much, it really belongs in a separate issue.
- The fact that MPL, fusion, and the "old" C++03 x3::variant exist might be unfortunate, but again, unrelated if all we want is X3 to work with std::variant/optional.
That being said, if you are working on a more thorough solution that supersedes anything that could come from this type of patches, great! Just don't forget about people using C++17 here and now, and want X3 to work with it, even if it's partially or slowly. The latter can be addressed in the form of a "support" header as @Lastique suggested.
In the end, perhaps a define enabling full-fledged use of the std facilities over the Boost versions would be preferable in this, but maybe that brings us to Spirit version 4 quite quickly...