http_get request command is not working

[nehaMittal18]

Hi Team, I am facing issue using http_get or http_post in ubuntu mariadb. i am following below url to setup http_get UDF as : http://open-bi.blogspot.in/2012/11/call-restful-web-services-from-mysql.html MariaDB [(none)]> SELECT http_get('curl google.com'); +-----------------------------+ | http_get('curl google.com') | +-----------------------------+ | NULL | +-----------------------------+ 1 row in set (0.01 sec)

[nehaMittal18]

build_vendor = unknown builddir = . datadir = ${datarootdir} datarootdir = ${prefix}/share docdir = ${datarootdir}/doc/${PACKAGE} dvidir = ${docdir} exec_prefix = ${prefix} host = x86_64-unknown-linux-gnu host_alias = host_cpu = x86_64 host_os = linux-gnu host_vendor = unknown htmldir = ${docdir} includedir = ${prefix}/include infodir = ${datarootdir}/info install_sh = $(SHELL) /root/mysql-udf-http/config/install-sh libdir = ${exec_prefix}/lib/mysql/plugin/ libexecdir = ${exec_prefix}/libexec localedir = ${datarootdir}/locale localstatedir = ${prefix}/var mandir = ${datarootdir}/man mkdir_p = /bin/mkdir -p oldincludedir = /usr/include pdfdir = ${docdir} prefix = /usr/lib/mysql program_transform_name = s,x,x, psdir = ${docdir} sbindir = ${exec_prefix}/sbin sharedstatedir = ${prefix}/com srcdir = . sysconfdir = ${prefix}/etc target_alias = top_builddir = . top_srcdir = . INCLUDES = SUBDIRS = src EXTRA_DIST = utils sql benchmarks all: all-recursive

OK, here are my makeFile variables. Do they help knowing installation path issues?? I believe it could be related to prefix and exec_prefix variable to be corrected while runnning: . /configure --prefix =/usr/local/mysql --with-mysql = /usr/local/mysql/bin/mysql_config make && make install

Please let me know if you need any other info to debug. There seems to be path differences for mysql installation for your manual and our mysql installation. I have done apt-get install mariadb-server and dont have package structure as your which seems expecting "/usr/local/mysql" installation and this mysql location does not exist on my machine.

Thanks a ton

[y-ken]

I'm afraid it supports only MySQL 5.1.x and 5.5.x. Recent 5.7 or later version does not supported.

[nehaMittal18]

is it working on maridb 5.5.x?

[asierra01]

@nehaMittal18 I think this is wrong SELECT http_get('curl google.com'); it should be SELECT http_get('http://www.google.com');

[czokie]

I tried it on MariaDB 5.5.56 that comes with Centos 7 - No luck. I also get NULL responses. Suggestions?

[y-ken]

It does not confirmed working at MariaDB currently. I didn't checked yet.

[czokie]

Ultimately, mariadb is the same codebase as mysql around UDF's - so it should work. I tried doing gets o http://127.0.0.1/ and confirmed it never reaches localhost web service... which obviously is why the response is null...

[czokie]

To help troubleshoot, I did a build on centos7 minimal and tried the provided install steps. I was unable to get it to link with lib curl (not sure why). Instead, I tried again and installed libcurl-devel. Install steps are as per below on a clean VM. Same errors I was having before. Interesting point, this does NOT appear to be related to the interface to MariaDB. It is not actually executing the libcurl request. I did try building and installing another UDF (the template one), and that worked just fine. So there is no problem with the UDF linkage to MySQL - The problem appears to be related to the lib curl side of things.

My install steps are listed below if anyone wants to replicate to help find an answer.

ulimit -SHn 65535
service mariadb start
git clone https://github.com/y-ken/mysql-udf-http.git
cd mysql-udf-http/
chmod 700 configure
./configure --with-mysql=/usr/bin/mysql_config --libdir=/usr/lib64/mysql/plugin
make && make install```

[czokie]

I have debugged a bit more - and this is what I found...

[pid 20035] [00007f5babe5970d] read(34, "\3SELECT http_get('http://192.168"..., 41) = 41
[pid 20035] [00007f5babe59884] fcntl(34, F_SETFL, O_RDWR|O_NONBLOCK) = 0
[pid 20035] [00007f5baa65146a] setsockopt(34, SOL_SOCKET, SO_RCVTIMEO, "\36\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 16) = 0
[pid 20035] [00007f5baa6514c7] socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 37
[pid 20035] [00007f5babe59884] fcntl(37, F_GETFL) = 0x2 (flags O_RDWR)
[pid 20035] [00007f5babe59884] fcntl(37, F_SETFL, O_RDWR|O_NONBLOCK) = 0
[pid 20035] [00007f5babe599ed] connect(37, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("192.168.1.254")}, 16) = -1 EACCES (Permission denied)
[pid 20035] [00007f5babe5976d] close(37) = 0
[pid 20035] [00007f5baa64185d] write(2, "error\n", 6) = 6
[pid 20035] [00007f5babe596ad] write(34, "\1\0\0\1\0017\0\0\2\3def\0\0\0!http_get('http:"..., 87) = 87
[pid 20035] [00007f5baa65146a] setsockopt(34, SOL_SOCKET, SO_RCVTIMEO, "\200p\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 16) = 0
[pid 20035] [00007f5babe5970d] read(34, 0x55ac15766720, 4) = -1 EAGAIN (Resource temporarily unavailable)
[pid 20035] [00007f5babe59884] fcntl(34, F_SETFL, O_RDWR) = 0
[pid 20035] [00007f5babe5970d] read(34,  <unfinished ...>```

Appreciate help on this :)

[czokie]

I think I found it - but don't know how to fix it. MySQL (or MariaDB) is starting via mysqld_safe ... which might be the problem. I manually logged in as the MySQL user, and started MySQL manually without going via mysqld_safe ... and it worked then.

Question is - What is the best method to safely start MySQL, while allowing this UDF to work?

[czokie]

Found / Fixed. Selinux was the culprit. Turned that off, and working just fine. I've not played with selinux before - but it would be good to know how to tweak it to allow MySQL to make outbound connections, without disabling it totally... Something to learn another day.

[czokie]

Spoke too soon. Better fix. The selinux command below will just tweak this one setting... This should be added to the README markdown page.

setsebool -P mysql_connect_any 1

[Neha412]

I executed this command by modifying the path

1)./configure --prefix=/usr/bin/mysql --with-mysql=/usr/bin/mysql_conf and below error get invoked

(cd .libs && rm -f mysql-udf-http.la && ln -s ../mysql-udf-http.la mysql-udf-http.la) make[2]: Leaving directory '/root/mysql-udf-http-1.0/src' make[1]: Leaving directory '/root/mysql-udf-http-1.0/src' make[1]: Entering directory '/root/mysql-udf-http-1.0' make[1]: Nothing to be done for 'all-am'. make[1]: Leaving directory '/root/mysql-udf-http-1.0' Making install in src make[1]: Entering directory '/root/mysql-udf-http-1.0/src' make[2]: Entering directory '/root/mysql-udf-http-1.0/src' test -z "/usr/bin/mysql/lib/mysql/plugin/" || /bin/mkdir -p "/usr/bin/mysql/lib/mysql/plugin/" /bin/mkdir: cannot create directory ‘/usr/bin/mysql’: Not a directory Makefile:256: recipe for target 'install-libLTLIBRARIES' failed make[2]: *** [install-libLTLIBRARIES] Error 1 make[2]: Leaving directory '/root/mysql-udf-http-1.0/src' Makefile:406: recipe for target 'install-am' failed make[1]: *** [install-am] Error 2 make[1]: Leaving directory '/root/mysql-udf-http-1.0/src' Makefile:248: recipe for target 'install-recursive' failed make: *** [install-recursive] Error 1

2)./configure --with-mysql=/usr/bin/mysql_config --libdir=/usr/lib/mysql/plugin . Making all in src make[1]: Entering directory '/root/mysql-udf-http-1.0/src' make all-am make[2]: Entering directory '/root/mysql-udf-http-1.0/src' make[2]: Leaving directory '/root/mysql-udf-http-1.0/src' make[1]: Leaving directory '/root/mysql-udf-http-1.0/src' make[1]: Entering directory '/root/mysql-udf-http-1.0' make[1]: Nothing to be done for 'all-am'. make[1]: Leaving directory '/root/mysql-udf-http-1.0' Making install in src make[1]: Entering directory '/root/mysql-udf-http-1.0/src' make[2]: Entering directory '/root/mysql-udf-http-1.0/src' test -z "/usr/lib/mysql/plugin" || /bin/mkdir -p "/usr/lib/mysql/plugin" /bin/bash ../libtool --preserve-dup-deps --mode=install /usr/bin/install -c 'mysql-udf-http.la' '/usr/lib/mysql/plugin/mysql-udf-http.la' /usr/bin/install -c .libs/mysql-udf-http.so.0.0.0 /usr/lib/mysql/plugin/mysql-udf-http.so.0.0.0 (cd /usr/lib/mysql/plugin && { ln -s -f mysql-udf-http.so.0.0.0 mysql-udf-http.so.0 || { rm -f mysql-udf-http.so.0 && ln -s mysql-udf-http.so.0.0.0 mysql-udf-http.so.0; }; }) (cd /usr/lib/mysql/plugin && { ln -s -f mysql-udf-http.so.0.0.0 mysql-udf-http.so || { rm -f mysql-udf-http.so && ln -s mysql-udf-http.so.0.0.0 mysql-udf-http.so; }; }) /usr/bin/install -c .libs/mysql-udf-http.lai /usr/lib/mysql/plugin/mysql-udf-http.la /usr/bin/install -c .libs/mysql-udf-http.a /usr/lib/mysql/plugin/mysql-udf-http.a chmod 644 /usr/lib/mysql/plugin/mysql-udf-http.a ranlib /usr/lib/mysql/plugin/mysql-udf-http.a libtool: install: warning: remember to run `libtool --finish /usr/bin/mysql/lib/mysql/plugin/' make[2]: Nothing to be done for 'install-data-am'. make[2]: Leaving directory '/root/mysql-udf-http-1.0/src' make[1]: Leaving directory '/root/mysql-udf-http-1.0/src' make[1]: Entering directory '/root/mysql-udf-http-1.0' make[2]: Entering directory '/root/mysql-udf-http-1.0' make[2]: Nothing to be done for 'install-exec-am'. make[2]: Nothing to be done for 'install-data-am'. make[2]: Leaving directory '/root/mysql-udf-http-1.0' make[1]: Leaving directory '/root/mysql-udf-http-1.0' root@udf-mysql:~/mysql-udf-http-1.0# cd ../ root@udf-mysql:~# mysql -u root -p For point second ,No errir get invoked .so I proceed the next step. open the Mysql console and create http_post method by executing following command create function http_post returns string soname 'mysql-udf-http.so'; -->function created successfully. Next I am trying to tweak Api by calling http_post as : SELECT http_post('http://m.baidu.com/s','word=xoyo&pn=0'); I got the below error : mysql> SELECT http_post('http://m.baidu.com/s','word=xoyo&pn=0'); ERROR 2013 (HY000): Lost connection to MySQL server during query

Version of my sql is : Server version: 5.7.21-0ubuntu0.16.04.1 (Ubuntu)

Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.

If anyone have any solution for this suggestions are welcomed.

[jasperkopalli]

has any one been successful in calling http request from mysql i have been searching sources but m getting error with commands i followed ste[ps from https://code.google.com/archive/p/mysql-udf-http/ m working on mysql 5.6 please help

[jhaoheng]

@jasperkopalli https://code.google.com/archive/p/mysql-udf-http/

and have to run below command in the mysql console, and it will build func table and data at mysql db.

mysql>

create function http_get returns string soname 'mysql-udf-http.so'; create function http_post returns string soname 'mysql-udf-http.so'; create function http_put returns string soname 'mysql-udf-http.so'; create function http_delete returns string soname 'mysql-udf-http.so';