Forums / Bugs, feature requests, and complaints
This discussion is an archived public thread. Posting and accounts are closed.
Forums: Post releases/WIPs without waiting for file convert?
Huddini · March 19, 2013 at 20:24 UTC
Hey, I noticed that songs don't actually get posted in the Relases/WIP section unless the server does its file converting thing when a user uploads his/her work.
While it would be inconvenient to just dl to listen to a track, is it possible to allow that bot to post a song in the forums, even when that file conversion is still in process? I wanted to give feedback to a user just now about his song I listened to on the main site, but right now I can't do it here since it hasn't been posted here by the bot yet.
Edit: Wait, the forums are already doing what I ask...but then why hasn't it posted this song yet?
http://remixsquared.com/remixes/3124-v2-ElectricMudkip-Wheres_your_Alibi_NowRe: Forums: Post releases/WIPs without waiting for file conv
quintin3265 · March 19, 2013 at 23:32 UTC
Thanks for the report. It was a bug. I fixed this issue, published the changes, and then rescheduled all the missed songs for conversion.
The cause was simple: I thought that the operation for determining whether a song was created before a certain date was LESS_THAN_OR_EQUAL. It actually is LESS_THAN_EQUAL, so the conversion routine was halting on an error every hour.
-------------------
There are three reasons that songs are delayed in conversion:
1. I think I already talked about the "system overload" reason for not converting songs immediately, see the other post for that one.
2. The user has to be given time to enter tags and games for the song. Otherwise, the forum post would contain the wrong text; for example, a song without games is an original song, but it also could have no games because the user simply hasn't had enough time to enter them yet.
3. I didn't want to draw attention to songs which have yet to be converted to streamable, because some people might not listen to them and never come back.
That said, it sounds like you disagree with reason #3. For now, I added your suggestion of a countdown timer, but I'm not opposed to making changes that people request, even ones that are significant. It would be possible to convert songs even more quickly, or change the "finished" button on the tags and games screen to schedule immediate conversion. But since that requires quite a while to accomplish, I'm wondering if the improvement (an hour per song) is worth the lost feature of, say, facebook integration, which could be implemented in the lost time.
Feel free to share your thoughts on this, and that includes other people as well.
Re: Forums: Post releases/WIPs without waiting for file conv
Huddini · March 19, 2013 at 23:32 UTC
quintin3265 wrote:
Thanks for the report. It was a bug. I fixed this issue, published the changes, and then rescheduled all the missed songs for conversion.
The cause was simple: I thought that the operation for determining whether a song was created before a certain date was LESS_THAN_OR_EQUAL. It actually is LESS_THAN_EQUAL, so the conversion routine was halting on an error every hour.
-------------------
There are three reasons that songs are delayed in conversion:
1. I think I already talked about the "system overload" reason for not converting songs immediately, see the other post for that one.
2. The user has to be given time to enter tags and games for the song. Otherwise, the forum post would contain the wrong text; for example, a song without games is an original song, but it also could have no games because the user simply hasn't had enough time to enter them yet.
3. I didn't want to draw attention to songs which have yet to be converted to streamable, because some people might not listen to them and never come back.
That said, it sounds like you disagree with reason #3. For now, I added your suggestion of a countdown timer, but I'm not opposed to making changes that people request, even ones that are significant. It would be possible to convert songs even more quickly, or change the "finished" button on the tags and games screen to schedule immediate conversion. But since that requires quite a while to accomplish, I'm wondering if the improvement (an hour per song) is worth the lost feature of, say, facebook integration, which could be implemented in the lost time.
Feel free to share your thoughts on this, and that includes other people as well.
Luckily I spotted the bug eh? Happy to help :P
-----
1. Yeah, totally understandable.
2. I'm not sure I follow. What "time-limit" do users have regarding this? Why is there a time-limit in the first place? Why doesn't the conversion start until AFTER the user enters up his/her tags? Unless I'm just not understanding you at all on this one :P
3. This is understandable too.
Indeed, there are the trade-offs to consider...How about a workaround solution (to where it might not inconvenience your dev time):
Allow submitter the option to have their submission posted to these forums AFTER they add their details (even though the file may still need converting), in case the user doesn't want to play the waiting game and wants immediate feedback from viewers (which as you indirectly pointed out, assumes the viewer is willing to not preview a track and just dl it quickly to listen instead).[/*:m]
Allow submitter the option to NOT have their submission posted to these forums, even after they add their details, UNLESS the file is converted all the way so it's streamable to viewers. This is for users who still want feedback, but are willing to wait so it's streamable for their users.[/*:m]
Allow submitter the option to NOT have their submission posted to these forums, no matter the other parameters considered. This is for people that obviously don't want feedback of any sort and are just using the site to upload their works for w/e reason.[/*:m][/list:o]
Good idea? No?
Also Facebook integration? Possibly a potential feature in the future or you were just really using that as an example?
Actually...How about a soundcloud integration (assuming they have any APIs to work with) where you can embed songs from soundcloud or something, so that...
Hey, yeah, is there a way you could make it so that if a user posts a song on soundloud, your bot could do that "auto-posting" by pulling info from their site, and then using THEIR embedded player to preview a song? (Because you know, it doesn't do that background converting like here).
Another solution I'm thinking of, is how about having a previewer for the original files a submitter uploads temporarily for songs still needing for conversion?
These are just ideas and all, but it's all I got :P
Re: Forums: Post releases/WIPs without waiting for file conv
quintin3265 · March 20, 2013 at 15:49 UTC
In regards to point #2, by "time limit," I'm referring to the timing discussed in a post in another thread. There isn't a limit to how often a user may modify games and tags. However, the song conversion always occurs at least one hour after the last game and tag edit, which is the "time limit" I was referring to. That delay ensures that the user isn't in the middle of editing games and tags, because the editing process is asynchronous. A user could enter a few tags, then leave the page open for fifteen minutes while he thinks about what to put in for a certain category, and then enter another tag. If there weren't a delay, then it's possible a forum post could be generated with incorrect information because the conversion routine would occur during that delay.
By the way, the reason that songs cannot be streamed before conversion is that the HTML5 specification is not implemented evenly across all browsers. For example, Firefox can stream *.wav files, but Internet Explorer can stream *.wmv files, and neither can do the other. The only format that is compatible with HTML5 in all browsers is mp3. And we need to stick with HTML5 because it's the only standard mobile devices support.
One thing that is interesting about this discussion is that, at YouTube, a much longer delay occurs while videos are converted. I guess everything comes down to the following question: is there a group of people who would be turned away permanently because they can't wait an hour and a half before others can listen? If you and others believe the answer is "yes," then I'm willing to invest significant time into changing the system's design, but we need to be completely sure that all that work is more important than all the other features that could be done in the meantime. So tell me the answer to this question, and I'll go along with what is said.
As to the social network integration, some of the features that were suggested include the following:
-Register by using your facebook account to fill in all the fields
-Auto-post remixes to facebook
-Auto-tweet
-Auto +1 your remixes with Google Plus
-Some sort of integration with linkedIn (more research would have to be done with this because I don't know enough about linkedIn)
-Your suggestion of integrating with soundcloud
There are other tweaks that are waiting, like:
-Adding a "register" button in the forums that redirects to the RS registration form
-Adding some sort of link to go back to the main site in the forums
-Making sure that the user is redirected to the previous page upon login
-In the "new songs" box, only display the latest song for each person, so that at least five people are always displayed there and one person can't flood the box
-Fix how the wrong posts are sometimes displayed in the "latest forum discussions" box
-The black text versus grey issue
I'd like some other people to prioritize both the tweaks and new features so I can get a sense of what is important or what should be skipped altogether.
As to soundcloud, I need to do more research on what API they provide, but there would be some concerns with the implementation you suggested. The major one is that if there is an "auto-import" feature, then it would pull in everything someone posts, even the songs that aren't game related, since we have no way of knowing which is which. Since soundcloud doesn't have games and tags like this, their songs would be uncategorized through an automated process. Finally, I doubt soundcloud exposes the source files, so we would have poor quality lossy or even lossy to lossy transcoded (!) mp3s.
Re: Forums: Post releases/WIPs without waiting for file conv
Huddini · March 20, 2013 at 18:24 UTC
Okay, WOW, alot to take in.
Okay, so it's not necessarily a time-limit, but also a delay. Does it really take the average user that long to figure out what tags they want to put in? The grace period seems pretty generous if I read you right.
Okay, I gotcha about the HTML5 one, .mp3 is the universal format all browsers can accept no problem.
-I think there is a potential for a group of people that will leave if they can't wait that long. The attention span of an Internet user is pretty short (and that's probably an understatement), since they expect to get all the info they need (that includes previewing/streaming features) in a snap of a finger.
However, I'd wait for other feedback on this particular matter since I'm not particularly confident in this claim.
As for the social integration, the most useful/convenient feature I'd reckon would be the "registering using facebook account" feature.
As for tweaks: It'd be cool if you could utilize part of the header as a navbar (I'm no web developer so correct me if I'm too wrong on the terminology), with a couple of links redirecting back to the main site, and for whatever available compo. As for prioritization, I'd say get the most easier tasks out of the way (which I presume is anything that deals with the forums)
As for soundcloud audio preview quality, you're right. They do this intentionally so a listener can't steal a song with its highest quality as to protect the uploader or something.
Re: Forums: Post releases/WIPs without waiting for file conv
quintin3265 · March 20, 2013 at 18:55 UTC
Hmmm, that's a problem. If soundcloud intentionally reduces quality, then that probably means they have some sort of disclaimer in their terms of service that prohibits your suggestion - and if we implemented it, someone would likely report that and the site would be broken. We should probably steer clear of that.
I do want to hear feedback, particularly from Shredster7 and DjTheS, on the "delay" issue. DjTheS has posted a large number of songs and he often links to them from other sites. It would be interesting to know if he simply waits the two hours before posting his link, if he just posts the link anyway without waiting, or whether he thinks this area is in need of a change. If he doesn't see this post, I'll send him an E-Mail requesting comment.
You might be surprised to learn that the stuff with the forums is the hardest stuff to change, because it's code that I didn't write. I'm not sure whether this used to be the case in the past or not, but most of my day is spent performing research to try to understand code and interfaces. I probably spend less than a third of the time actually writing code; software has become so complicated that two-thirds of the time is spent trying to figure out what other people have written. The forum issues like trying to add a "register" button that goes to the right place are much more difficult than if I had to make a change to the main site because hours are required to research the phpBB code.
For example, I've been working on the issue where users are logged out of the forums but not the main site for four weeks and still haven't found an answer to it. Yet, I think I would be able to implement another commonly requested feature, zipped archives of an artist's songs, in just a few hours.
It would be possible to eliminate the whole top bar of the forums altogether and replace it with the main site navigation bar. Since Shredster has volunteered to take over administering the forums, I'll defer to him on whether he likes that change and how he thinks the forums should look.
Re: Forums: Post releases/WIPs without waiting for file conv
Huddini · March 20, 2013 at 18:55 UTC
quintin3265 wrote:
Hmmm, that's a problem. If soundcloud intentionally reduces quality, then that probably means they have some sort of disclaimer in their terms of service that prohibits your suggestion - and if we implemented it, someone would likely report that and the site would be broken. We should probably steer clear of that.
I do want to hear feedback, particularly from Shredster7 and DjTheS, on the "delay" issue. DjTheS has posted a large number of songs and he often links to them from other sites. It would be interesting to know if he simply waits the two hours before posting his link, if he just posts the link anyway without waiting, or whether he thinks this area is in need of a change. If he doesn't see this post, I'll send him an E-Mail requesting comment.
You might be surprised to learn that the stuff with the forums is the hardest stuff to change, because it's code that I didn't write. I'm not sure whether this used to be the case in the past or not, but most of my day is spent performing research to try to understand code and interfaces. I probably spend less than a third of the time actually writing code; software has become so complicated that two-thirds of the time is spent trying to figure out what other people have written. The forum issues like trying to add a "register" button that goes to the right place are much more difficult than if I had to make a change to the main site because hours are required to research the phpBB code.
For example, I've been working on the issue where users are logged out of the forums but not the main site for four weeks and still haven't found an answer to it. Yet, I think I would be able to implement another commonly requested feature, zipped archives of an artist's songs, in just a few hours.
It would be possible to eliminate the whole top bar of the forums altogether and replace it with the main site navigation bar. Since Shredster has volunteered to take over administering the forums, I'll defer to him on whether he likes that change and how he thinks the forums should look.
Well for the soundcloud, they still allow embedding, they just prevent actual download of a song (which is up to the soundcloud user).
But better to be safe than sorry and check out their terms I guess.
I think they'd be better, seeing as they (at least DJscot because that's the only guy I recognize) seem to be a better representation of musicians here and understand what the needs are for other music-making folks
As for the logging out thing, you mean something like this?
https://www.phpbb.com/community/viewtopic.php?f=71&t=1057885And the register thing?
https://www.phpbb.com/community/viewtopic.php?f=74&t=1480745https://www.phpbb.com/community/viewtopic.php?f=74&t=1710705Annnd for redirect after login?
https://www.phpbb.com/community/viewtopic.php?f=46&t=981915I don't know if it'll be useful/helpful to you, but it's a start.
As for the navbar, well, we'll just have to see the input of the admin then ;)
Re: Forums: Post releases/WIPs without waiting for file conv
quintin3265 · March 21, 2013 at 15:08 UTC
I sent an E-Mail to DJtheS to see if he would be interested in replying here.
Thanks for the heads up with those links. I had originally planned to spend this weekend tilling the garden, putting down mulch, washing the salt out of the garage and off the cars, and other similar activities. But with temperatures at 25 degrees and snow still covering the ground, I will instead spend a lot of time working on improving the site. I'm hoping to get feedback before tomorrow so that I can decide what to work on.
As to soundcloud, the problem with integrating with other sites is that often, other sites make "improvements" without considering the impact on other sites that do not need them. I stopped developing stuff that works with eBay because they constantly changed their API with new seller policies and deprecated functions.
Generally, you have a program that is finished and works, and you don't need to add anything to it. But then many other sites force you to make changes to deal with new features that you could care less about. Therefore, if the remixsquared architecture with delayed song conversions is a problem, then we should do it right and spend the time to change this site, not implement a workaround with soundcloud. Even if it takes longer now, we'll have a better and more consistent system. More importantly, we won't end up in a situation where soundcloud changes its API and there is no manpower available to retest everything on short notice. We'll see what djthes and shredster have to say about this if they reply to their E-Mails.
Re: Forums: Post releases/WIPs without waiting for file conv
DjtheSdotcom · March 21, 2013 at 16:58 UTC
Well, I personally don't have a problem with the delay, I usually add the link right away even if it hasn't finished converting yet. Nobody has complained about that yet, some even download the *.wav file. Since they already know from the video how the song sounds, they don't need a preview.
I'd like to add that I always add another mp3 download link, the link to remixsquared is always labeled as "best quality download".
Re: Forums: Post releases/WIPs without waiting for file conv
Shredster7 · March 21, 2013 at 22:35 UTC
Regarding the various issues discussed in this thread, here's my thoughts...
1) What format are the songs being converted to anyways? If it were possible to upload that format, would it be possible for the song player to play that uncompressed version of the song instead? For example, if the necessary format is .OGG, then if I uploaded "Song X" in OGG format, would the player be able to play the song while the converter cleans up unnecessary metadata?
2) Having the extra options dictating when the NewSongBot uploads songs actually doesn't sound too difficult, provided all of it is handled on the main site. Judging by what Quint is saying about modding the forum's code, if any of the work required modding the bot's behavior on the forums, I'd save the task for later on, after other optimizations to the site are finished.
3) I have no input on the Facebook situation. I honestly try to stay away from it.
4) Soundcloud integration just seems unnecessary to me. As Huddini pointed out, Soundcloud's habit of lowering sound quality intentionally, especially for a quality snob such as myself, would be ridiculously irritating. Not to mention, the idea of having a music player that clearly belongs to another site would altogether cheapen the 'feel' of a site as a whole. People tend to hang around well-polished sites longer than a patchwork of plugins/add-ons/open source code/etc.
5) All of the this:
-In the "new songs" box, only display the latest song for each person, so that at least five people are always displayed there and one person can't flood the box
I've seen this exact scenario happen two or three times since I've joined the site, and depending on how my other pet projects go,
I could end up doing that. I don't want to push other new songs completely off the radar. Imagine a new user uploads his first truly great remix, and discovers that his song has been wiped from the new song feed the next day. It's downright infuriating.
6) Having the main site's header bar in place of the current
pre-made stock header (hint, hint) would probably be a massive improvement. Again, it'd make the site feel much more professional and it'd make it many times more convenient to switch between the main site and the forums (especially after hitting the "Discuss this song in the forums" button).
There was something else I wanted to add, but I just plain cannot remember what it was. Dang it!
Re: Forums: Post releases/WIPs without waiting for file conv
quintin3265 · March 22, 2013 at 00:17 UTC
Songs are converted to MP3, but even if the song is already MP3, the "conversion" routine also needs to obtain metadata about the song, like how long it is. Right now, the same routine determines if the song is an MP3 or some other format, gets the metadata either way, and converts to MP3 if the original song is some other format. Without that metadata, the streaming player wouldn't be able to show a progress bar or any player controls. More seriously, users could upload executables or other bad things that have a .mp3 extension - that's why we need to run the songs through the "conversion" function (which really does more than converting) first.
I think the "new songs being flooded" issue should be relatively easy to fix. That was actually something I had always planned on doing, once the site became big enough to actually need it. Right now, it's just on the cusp of that issue being a problem. After some thought, I thought that the best algorithm would be the following:
1. Query the latest version of a song that all users have uploaded.
2. Sort the songs in descending order by date.
3. Display all songs posted today.
4. If there are fewer than five songs that meet the criteria at this point, then go back as many days as necessary until five songs that meet the criteria are found.
Now that DjTheS and Shredster have responded, I think that most people agree that the "soundcloud" issue is a good thing to avoid. I think that the solution is to add a new field to "song" and "song_latest_version" called "ready_for_conversion." The "Finished" button on the tags screen would be changed to set this "ready_for_conversion" equal to true. There would then be a second process job added: the new job would run every hour and set "ready_for_conversion" equal to true for the people who forget to press the button. While the "song" table already has 50 columns in it, adding one more boolean value shouldn't have any significant effect on everything else.
The problem with displaying the main site's header bar in the forums is that the main site header bar is fixed-width, while the forums are variable-width. It is possible to make the forums fixed-width, of course, but that looked weird when I tried it. I'll work on this tomorrow and try to display it on the test site. But I want to make some changes to the forums this weekend that Shredster suggested to support his efforts to improve readership.
I also try to stay away from facebook, simply because by the time I get home after 12 hours at work and the gym, I really don't care about people's cat pictures. I want to spend time relaxing or meeting people in person, not posting about my life on facebook. But there are a lot of people who disagree with me, so we do have to consider them at some point (just probably not now).
Re: Forums: Post releases/WIPs without waiting for file conv
Huddini · March 22, 2013 at 18:58 UTC
Hey unrelated to this particular topic, but can you have an alternate dark theme available for the forums? (Not to replace the one you have now, but just as a second option).
I don't particular dig the bright colored theme going on with the main site/forums, but that's because I generally prefer darker/blacker colors (because it's easier on my eyes).
And you don't need to necessarily use the main site header navbar w/e, just supply links within the current forum header bar now. No one's going to particular care what the navbar looks like so long has the links are there.
Re: Forums: Post releases/WIPs without waiting for file conv
quintin3265 · March 23, 2013 at 01:38 UTC
OK, I spent most of the day today and was able to change a lot of things in the system to address the song conversion issues. Here is the new way things will work as soon as these changes are deployed:
1. The "Finished" button at the bottom of the "songs and tags" screen has been renamed to "Process song."
2. If the user enters tags and then leaves the site, or if the user enters no tags at all, the song will be converted by a "slow" conversion process, which is there simply to ensure that people who forget to click the "process song" button don't end up with songs that are never converted. This "slow conversion" process will run every 12 hours.
3. If the user enters tags/games and then clicks "process song," it is submitted to the "fast conversion" process, which runs every five minutes.
4. The song details page explicitly reminds users who have not clicked "process song" that their conversions will be delayed by 12 hours until they have clicked it.
This should address the best of both worlds: allowing those who want instant gratification to have their songs converted in under five minutes, while also making sure that people who forget or who need more time aren't left behind.
I have not yet addressed the forums; I'm going to work on that tomorrow. I think that the main navbar is the solution to almost all of the forum problems - for example, the "missing register button" issue is resolved by it, as is the "lack of unified theme" issue.
As to the color scheme, I really appreciate most advice given here, and I think it would be a great idea to have a choice of color schemes, but the amount of work to do that would be astronomical. Without the help of additional developers, I don't think it would be possible to support multiple schemes. It is possible to make tweaks to the current scheme, though, and if people can agree on them, then I can make them.
I also came to the realization that the reason there are so many bugs at the worst possible time (Mondays) is because I usually work on the site on weekends and release the changes on Sunday. Then the bugs crop up Monday morning and can't be fixed for a while. So from now on, I'm only going to release on Saturdays, and hold changes made on Sundays until the next weekend unless they are major bugfixes.
Re: Forums: Post releases/WIPs without waiting for file conv
Huddini · March 23, 2013 at 05:20 UTC
The whole fast conversion/slow conversion idea is pretty good solution! As well as naming to "Process song"
Also, as for the theme, editing the color scheme for this particular thread style should be just enough (just imo phpbb regular theme is pretty lame looking in general, migh as well change the forum section icons while you're at it :P), I think theyre called child themes you can base off of whatever parent. This is just a preferable option o work on though.
Good point about when bugs show up. So we wait every Monday until any srs bug might show up :P
Thanks for taking your time on trying to implement our feedback without being an ass about it btw, devs such as yourself seem very hard to come by the
internet nowadays :)
Re: Forums: Post releases/WIPs without waiting for file conv
quintin3265 · March 24, 2013 at 16:32 UTC
Many of the forums issues have now been addressed. There are two improvements I think that are still outstanding: the logged-in user's profile picture could be displayed in the empty space in the upper right, and all the statistics present on the main site could also be displayed there. I did not make changes to the text color because that is a major change that I want more input on.
Re: Forums: Post releases/WIPs without waiting for file conv
Huddini · March 24, 2013 at 21:22 UTC
Wait, what text color for the forums? They seem fine to me atm.
Edit: Oh hey, nice job with the navbar!
