Support for RETS to import MLS data

[etewiah]

The number 1 feature request I have received since launching PropertyWebBuilder has been to add support for importing MLS data. This is on my TODO list and I'm looking at various options such as:

• Connect directly to MLSs that support RETS using a gem such as https://github.com/josephholsten/rets4r or https://github.com/estately/rets
• Use a third party intermediate service like https://retsrabbit.com or https://simplyrets.com
• Create my own huginn agent (https://github.com/cantino/huginn) to use as an intermediate service

The third option is likely to be the most difficult and time consuming but it is also the one that most appeals to me. Huginn is an amazing tool and using it would open up the possibility to accommodate new endpoints for importing data more easily. I will investigate it some more over the coming days and update this issue on the progress I make.

If this feature is of interest to you (either as a developer or a realtor) feel free to comment on this issue.

[jwoertink]

This project looks pretty cool! I wrote a gem a while back https://github.com/jwoertink/tourets that could be good for it. I don't work in this field anymore, so I haven't had the chance to really work on it. If you're interested in it, I can make you a maintainer on it, and help out where I can.

[etewiah]

@jwoertink thanks for reaching out - great to hear from you!

Just had a look at tourets - it looks ideal for my use case. Right now I'm still busy fixing some bugs, adding tests etc. I hope to be able to start work on RETS integration in the next couple of weeks or so. At that point I will give tourets a spin and see if it works out for me.

The only thing I see right now that might be an issue is that its built on the placester RETS gem which is not being maintained anymore. If its fairly easy to swap that out for the estately rets gem (https://github.com/estately/rets) then I'll try to do that.

Yes, please make me a maintainer and I'll try to contribute.

One more thing - did you happen to find some RETS servers against which I could try things out with test credentials?

[jwoertink]

My master branch is way out of date, but I have a rails branch that already has the placester gem ripped out. As far as I know, you have to pay to have access to the MLS and RETS. The idea behind TouRETS though was for an easy rails integration while being able to support multiple RETS server.

I'll add you on there, and we can work on making it more solid. Maybe @russ would like to get in on this action?

[etewiah]

Thanks @jwoertink - got your invite. I'll look into it as soon as I get the chance.

[etewiah]

Just found out today about an initiative to make it easier to access MLS data for development purposes. Applying for access now. Hope it makes things easier.

http://www.inman.com/2017/01/26/free-mls-data-feed-could-speed-up-real-estate-innovation/amp/

[mcohen9]

Just saw the updates for importing data from the MLS. looking pretty good so far this is exactly what im looking for. A suggestion would be to add support for OLR imports since its the main DB used for the NYC real estate market.

[etewiah]

Hi @mcohen9 , that was fast - I haven't even got round to writing about it here ;)

I have just had a look at the OLR website and they don't give me a chance to create a test account. Can you provide me a test account that I can use for development purposes? It can be just a temporary one that is deleted after a few days.

[mcohen9]

Thanks man! @etewiah yes whats your email?

[etewiah]

No worries @mcohen9. Just send me an email (ed at propertywebbuilder.com) and let's figure out a way to make it work.

[PaulyStix]

Hi,

I am currently in Albany,NY using https://crmls.paragonrels.com/ParagonLS/Default.mvc. Any help getting this added would be great!

[etewiah]

Hi @PaulyStix , I will have a look sometime next week. Do you have test credentials or a test server somewhere I can use for developing with?

[PaulyStix]

Hi Ed,

Here is the link to the MLS we use. Also is there a way to edit the website name to take out the herokuapp part? We have a domain name and I wondered if that is possible to use instead.

https://crmls.paragonrels.com/ParagonLS/Default.mvc/Login

On May 12, 2017 11:58 AM, "Ed Tewiah" [email protected] wrote:

Hi @PaulyStix https://github.com/paulystix , I will have a look sometime next week. Do you have test credentials or a test server somewhere I can use for developing with?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/etewiah/property_web_builder/issues/2#issuecomment-301116294, or mute the thread https://github.com/notifications/unsubscribe-auth/AMevEScswT0cVGjNwwcmIi7W-UsdryILks5r5IGngaJpZM4LYv47 .

[etewiah]

Hi @PaulyStix, heroku does indeed allow you to add custom domains for free. The instructions are here:

https://devcenter.heroku.com/articles/custom-domains

If that is too confusing, give me a shout and I'll talk you through it. (BTW - for each question, it is better to open a different issue. This will make it easier for people who have the same question in future to find the answer)

For the MLS, you already gave me the url but as I don't have credentials (username / password) to access it I cannot test to see exactly how the data is returned. Often an MLS will let you create test credential which you can share with a dev. Send me an email (ed at propertywebbuilder.com ) and we can discuss further.

[PaulyStix]

Good Morning Ed,

I don't think I create a test user but I still looking into it. We could always share a screen and I could let you got through the system if that would help.

Thanks, Paul ᐧ

On Fri, May 12, 2017 at 2:55 PM, Ed Tewiah [email protected] wrote:

Hi @PaulyStix https://github.com/paulystix, heroku does indeed allow you to add custom domains for free. The instructions are here:

https://devcenter.heroku.com/articles/custom-domains

If that is too confusing, give me a shout and I'll talk you through it. (BTW - for each question, it is better to open a different issue. This will make it easier for people who have the same question in future to find the answer)

For the MLS, you already gave me the url but as I don't have credentials (username / password) to access it I cannot test to see exactly how the data is returned. Often an MLS will let you create test credential which you can share with a dev. Send me an email (ed at propertywebbuilder.com ) and we can discuss further.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/etewiah/property_web_builder/issues/2#issuecomment-301158005, or mute the thread https://github.com/notifications/unsubscribe-auth/AMevEdCAQkebdeiOUesMXkK7ZDXys3joks5r5KslgaJpZM4LYv47 .

[etewiah]

Hi Paul, sharing a screen won't help much I'm afraid. When I get the chance I'll approach the MLS myself to ask if they can provide some test/dev data I can work with.

Did you have any luck with the custom domain on heroku?

[PaulyStix]

Hi Ed,

I managed to get this far with the domain. (See attached) I am not sure how to configure the DNS to point at the target but I'm still trying.

Is there anything I can do on my end for the MLS? If I login can I pull the code on the page to maybe help you?

Thanks, Paul ᐧ

On Thu, May 18, 2017 at 7:45 AM, Ed Tewiah [email protected] wrote:

Hi Paul, sharing a screen won't help much I'm afraid. When I get the chance I'll approach the MLS myself to ask if they can provide some test/dev data I can work with.

Did you have any luck with the custom domain on heroku?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/etewiah/property_web_builder/issues/2#issuecomment-302380660, or mute the thread https://github.com/notifications/unsubscribe-auth/AMevEcS4a5fXREyXZpkBnJQRxILB7VGBks5r7C88gaJpZM4LYv47 .

[etewiah]

Hi Paul, let's continue this conversation by email - everyone subscribed to this issue will get notifications about each comment here which can get annoying for them. I can't see your attachment so please email it to me ( ed at propertywebbuilder.com ) and I'll try to help.

[Jscott388]

Where you guys at with this? I have access to RETS data. I have 520 fields that I have to deal with. They aren't clearly. Some just say LM_Char10_7 as the field, some have better names, but over all the field labeling is ugly. So once you get this data, I am like how do I want to save this, in my case I am using mongodb. From there I have to figure out how to keep this updated. This is no easy task.

[etewiah]

Hey @Jscott388, I haven't had the chance to work on this in the last month. I hope to be able to return to it in a couple of weeks though. I am also thinking of making a standalone project out of this functionality. That would make it easier for people to export to their own data stores like mongodb.

Yes, one of the tricky bits to this is getting all the field mappings right. Do you have any Ruby on Rails experience? If you want me to explain a bit about how I went about this, feel free to give me a shout on skype (my user id is tewiii ) and we can schedule a chat.

[Jscott388]

I figured it out. Use a mongoose schema and set the alias as the system name from the Rets data.

[christmoore]

What's the current status for this issue?

[etewiah]

Right now my focus is on migrating to vue js with the quasar framework:

https://github.com/etewiah/property_web_builder/discussions/131

Happy to return to this issue if someone wants to work with me on it.