miguelfreitas
Possible to add a dedicated url field ?
Hi,
Is it possible to have an url field outside of text content ?
140 chars is very short and using url shortner add dependency to an external service... It's a pity and in contradiction with a so decentralized architecture :-(
Thanks for your opinion about this idea.
Yes, it is possible. But how do you think about the UI for that? One possibility is that the UI would identify the url, remove it from the message and add it to the extra field. However when one forwards the post to a legacy system the URL wouldn't be visible. Is this a good compromise?
I didn't think about multicast posting on several systems. Do you think that adoption of Twister could be conditioned by content compatibility. If you do, it's important to keep this compatibility.
Legacy system automatically replace long url with a short one, so long urls do not consume many characters in text. Keeping whole url in text for Twister is a huge limitation for user. In the other side, how Twister should manage legacy content with legacy shortened urls ? Has it to keep this ugly short url inside text ?
More generally, a dedicated field would permit to escape from every legacy url shortners and maybe get target url when pasting a shortened one.
When forwarding to a legacy, tool (addon, UI or other) could concatenate text and url, and let legacy system shortening it its way. Friendica addon does it. Maybe Twister UI char counter could change color (orange) when it detects that text + legacy shortened url is too long to prevent user.
I also think a short url mechanism is needed. It could handle both urls added to a post/dm or stand alone urls. URL shorter centralization is a huge problem.
Could something similar not be used to build in something like tweetmore?
Considering Twitter actively forbids cross-posting with other networks, I'd say to hell with compatibility. The lack of proper metadata fields in twitter has always been one of its drawbacks, let's not repeat it here. A URL field would be perfectly fine, if it's not too technically difficult to implement.
I'm not aware of this cross-posting prohibition.
Are you saying that if I want to duplicate my own post in two microbloggings they would forbid me of doing so? Sounds absurd. I can't give up on the copyright of my writting, neither they can claim exclusive copyright over my own posts.
In order to post on Twitter via API you have to authenticate with OAuth, which is only possible for Twitter registered applications. One of the clauses in applications' standard TOS is that you will not use the app to synchronise with other social networks, end-of-story. This is a policy they implemented in the last 2 years as part of their big "crush 3rd party clients & other social networks" strategy -- they want people to use Twitter official clients and keep Twitter content inside Twitter. Of course you could register a "personal" app that would surreptitiously post your messages to Twitter and Twister, but if they found out, they would disable it (and potentially ban you). All big services that offered Twitter bridges (IFTTT etc) had to remove them.
This is insane. They can't claim copyright over my own posts: if I copy-paste my post into this app using Twitter API and then also copy-paste it into my other twister app that would be legal, right?
Btw, what about just reading posts from Twitter (not posting)? Do we need to sign a TOS as well?
Yup, they disabled public RSS feeds, so you need an API key to do pretty much anything. And sure, you maintain copyright and can do copy-paste, what they won't allow is to do it programmatically via their API. One could use a scraper, but you can imagine how painful that is.
This is why nobody does any Twitter integration anymore, except for the occasional post-update-to-Twitter from non-social applications.
Currently, I have an IFTTT rule that scrapes my Twitter feed and reposts any new posts to my Google Plus page (via BufferApp.com). But as toyg points out, I had to approve the app via their API first. To my mind, there shouldn't be a separate field for the URL (Twitter doesn't do that), but there should instead be some sort of URL-shortening taking place. Unfortunately, that would require DNS involvement to shorten a URL, so I can see why it hasn't been implemented. Still, it's something for us to be thinking about in my opinion. Perhaps there is some decentralized way of shortening links that only someone running twisterd can see the shortened link? Like a decentralized mini-dns system I'm imagining here. Kind of like what TOR does with their http://xxxxxxxxxx.onion/ URLs. So perhaps there could be http://xxxxxxxxxx.twister/ URLs?
@ezrafree A simpler option not to have to deal and highjack the DNS might be to add a URL handler? shorttp://
@ezrafree if we want a decentralized url shortener inside twister network we can easily add new DHT resources for it. For example: ./twisterd dhtput ezrafree shorturl_123 s "http://long-url-here" The short link will need to be tied to your username though, perhaps using a different protocol like @BlockTester said: shorttp://ezrafree.123 The UI may convert it to the right link on the fly. Interesting ideas! :-)
@toyg Did you see https://github.com/Astalaseven/twitter-rss? It is a scraper which works for my private needs, although it is not running perfectly stable.
@miguelfreitas Good idea! it would be nice if you do this my advice short url format: shttp://%SHORTURL%.%username% Example% shttp://AffA3c.shift/ = http://google.com/blablabla
If a url shortener is implemented in twister, it would be nice te be able to reduce any kind of urls (http://, ftp://, sftp://, irc://). It shouldn't be hard to had the scheme to the shortened url. For recall, the syntax of a url is scheme://domain:port/path?query_string#fragment_id
So far, I like the idea of @miguelfreitas best, but I'd suggest 3 modifications
- Since we're shortening, let's make the protocol short:
sh://123.mfreitas(putting the user as "tld" is @[twister]rysiek's idea, and it makes sense). - We can decide that the "default tld" is the author, so you could say
sh://123and save 9 letters :spades:. - If we do
./twisterd dhtput ezrafree shorturl_123 s "some descrptive text|http://long-url-here"then the client has something to display as the link text or tooltip (although this might be a bit tricky for NoJS :wink:).
@nitmir I think we should allow them all (BTW, about time someone started shorting magnet links), but it should be a well-known whitelist. We know we don't want javascript:// but what about cam:// (or ejectorseat:// :stuck_out_tongue_winking_eye:)?
OHAI, so I'm out of the woodwork.
- Anyway, I think I wouldn't go for the "default TLD, you don't have to use it if it's your link", as that would make some twists non-reshareable: suppose I make a 139-char twist with a sh://123; when somebody wants to re-share it, the link becomes sh://123.rysiek - and that gets the twist over 140 chars. So I'd say: use the TLD explicitly, always.
- As far as #NoJS clients are concerned, I guess it could be solved thus:
- get twist text
- if the twist contains an sh:// link, get the full link and replace it in the text
- display the twist with the replaced link.
- Do we want the 'sh' there? This might be non-transparent for new users, esp. those with some shell scripting experience. Maybe "link://" or "l://" (that's "small-L" there)?
So I'd say: use the TLD explicitly, always.
You've got me convinced.
if the twist contains an sh:// link, get the full link and replace it in the text
If we do it client side, it could take forever (this is why swizzler now fetches user details as iframes, and on the main page they're only mentioned as @usename). The way around this would be that the link opens /shorturl/thedod/123 in the search iframe. It shows long urt as a link, and — if @miguelfreitas implements it :angel: — the descriptive text that has been dhtputten (is that a word?) with the url.
Maybe "link://" or "l://" (that's "small-L" there)?
I think lowercase-L is an amiguity we can avoid by using u:// (u is short for "url-shortener-generated short url for use in string-length-constrained environments (e.g. Twister)" :innocent:)
@dryabov had an idea to couple the shortened links to a specific twist (dhtput them as an array). This got me thinking, why not a dict, and then it's zero markup? You write "my site has a new lolcat" and then a dict tells the client to convert "my site" and "lolcat" to links
In fact, perhaps this can be done as part of the dhtput of the post itself. I'll go experiment a bit. I hope I won't break anything :smiling_imp:
update: to do this - we'd need to rewrite createSignedUserpost() in twister-core. Not my department :stuck_out_tongue_winking_eye:
Oh this is a perfect way of dealing with it! No crazy url schemes, no extra characters like 'u://', etc. The only problem I see is when somebody posts a long link from a pure text client -- it should be changed into something short.
also, how would post editing look like? where do I write the dict data and should I use the {...} structure? this will be unusable for regular joes and jannettes, I think...
The only problem I see is when somebody posts a long link from a pure text client -- it should be changed into something short.
Why is that? The url never gets to be part of the twist. It's just an "overlay". In an SMS (for example) it would look like plaintext (maybe additional sms per link: lolcat: htt.... in case your phone can use that). BTW, SMS is no longer limited to 140 (even where I live :cat: :bear: :tiger:).
also, how would post editing look like? where do I write the dict data and should I use the {...} structure? this will be unusable for regular joes and jannettes, I think...
Heavens forbid. editor would roughly look like
Message: [I got a dog without a nose ]
links:
[a dog ][http://doge.orgy ]
[nose ][magnet:..... ]
<add a new link>
Before fancy js editors were invented, such an interface was common for talk-backs in newspapers etc. (I don't think wordpress existed yet). I.e. end users (newspaper readers) understood how to use them
In the end it all boils down to [a refined version of] @jpfox's initial request. I propose an extended version of twisterd newpostmsg thedod 123 'the rain in https://en.wikipedia.org/wiki/Spain':
twisterd newlinkedmsg thedod 123 '{"msg":"the rain in Spain","links":{"rain":"...",...}}'
The userpost would look like unlinked text to an "old-skool" client (still has msg), and future clients would know to linkify it using links
{"userpost" : {
"height" : 33103,
"k" : 389,
"lastk" : 388,
"msg" : "The rain in Spain is case-insensitive",
"links" : {
"rain":"https://en.wikipedia.org/wiki/Rain",
"spain":"https://en.wikipedia.org/wiki/Spain"},
"n" : "thedod",
"time" : 1397509203}}
Security heads up
We should have a whitelist of allowed protocols (not "anything but javascript:", because some devices have protocols for camera etc. so why guess what to blacklist?). We should also set limits on url size, number of links, etc. to avoid DoSing with a 1GB twist :smiling_imp:
@thedod currently i'm more interested in having an extended version of newpostmsg where raw "userpost" contents may be passed for signing and propagation. Or, better yet, receiving already signed content as mentioned in #4 to allow full web-clients solutions... that would be a huge win for widespread usage.
@miguelfreitas mobile clients with twisterd is better for networks than client-server architecture I hope that in the future we find IOS/Android developer that can write native client for that platforms (not web app) than push app to stores.
@miguelfreitas I don't see why we need to add JS encryption and weaken the system unnecessarily, and anyway - what would swizzler and curses clients do?
I think that twisterd should handle high level signing of json (as newpostmsg already does), and not count on clients to construct messages according to the protocol (they may be outdated, or buggy, or pwned).
I'd even consider removing signmessage from the rpc interface and only allow high level operations like "post" or "change profile", because the only way to handle unknown attacks is to whitelist what we define as acceptable.
For posting a twist, we already have newpostmsg that accepts a string payload, and I've proposed newlinkedmsg (including analysis of threats I could think of) as a possible alternative, but it could be anything else (e.g. call it newstyledmsg and also have fields like "bold":["call","have fields"]). The important thing is to keep creation of a signed message inside the twisterd box, where we can try to stop all known atacks (enforce size limits, avoid javascript: links, etc.) instead of counting on all clients to be up to date about it.
@thedod you are thinking security the wrong way. adding json validation rules to twisterd when creating posts is useless. I own my twisterd client and i can do anything to it's code, including the removal of these validation rules. the only validation that really counts is the one you apply to the data that you receive from network, this is where stuff needs to be sanitized.
never trust that your attacker is running a good and well behaved twisterd copy that validated all json fields for you.
about having the JS encryption or not: i believe this is about whether we want to keep twister for a small group of hackers and users who install their own software, or if we want to reach the other 90% of users who will only access twister if it works inside their browser. of course we shouldn't trust on a site to deliver a good JS for you, so up to this point, the only solution i can think is installing a chrome app or similar. not as good as not installing anything, but at least we could be sure to be running a software that is authenticated... that's a different discussion though.
@thedod if we had "change profile" as a written-to-stone twisterd RPC, now we wouldn't be seeing the tox and bitmessage fields that Calm theme added. i believe twisterd should help you, not prevent innovation and new ideas...
never trust that your attacker is running a good and well behaved twisterd copy that validated all json fields for you.
If there's a single "bad" client trying to post a post that doesn't comply with standards, I'd expect the "good" clients not to accept it. If this is not being done today, what's keeping an attacker today from posting 1GB twists and sinking the network down?
I certainly wouldn't want to reach a state where all known attacks should be handled by all known clients.
if we had "change profile" as a written-to-stone twisterd RPC, now we wouldn't be seeing the tox and bitmessage fields that Calm theme added. i believe twisterd should help you, not prevent innovation and new ideas...
Well, they got lucky :wink:. As a counter example, you now have this request (where there's no way around it and it would require twisterd code changes). Wouldn't I like things to be easier for developers? Sure, but not at the expense of security. What would happen if profiles were "written in stone?", this would probably come as a pull-request, and would end up as a framework that makes it easy to pull-request the next one (top 10 recommended magnets?).
about having the JS encryption or not: i believe this is about whether we want to keep twister for a small group of hackers and users who install their own software, or if we want to reach the other 90% of users who will only access twister if it works inside their browser.
Why should end users care whether encryption was done in javascript or not? (also, what makes you think a NoJS web app can't be user friendly or that gateways to mail or im can't reach users in a "friendly" way, but even if "user friendly requires JS". Where does it say "JS encryption"?)