Hi, I'm the same age as you, born in the UK but have been working in USA for a few years now. I am also feeling burned out by my job, spend a lot of time fantasizing about retiring early.
This is a shot in the dark but I feel like a lot of my mental state has been caused by covid, missing regularly seeing my friends, family and the alienating nature of interacting with my colleagues only via screen. Because of this I have resolved to not make any large decisions until covid is completely over, since it's hard for me to assess how much differently I will feel once things get back to normal. Till then I save as much as possible to give myself more options.
I'm guessing that since there are hundreds of millions of repositories the typical marginal value of someone's contributions would optimistically be on the order of a few dollars. But since the consensus on HN is that they spend very little time actually coding and there is no use-case for copilot, perhaps it worth a lot less.
If I stole just $0.50 from every american, the typical marginal value of their contribution is tiny, but I still stole nearly $200M. Maybe none of those people will raise much of a stink because it's just $0.50, but it's just as bad.
Practically, it's bad in that I never got the chance dictate how they use my code. My GPL code has very little
marginal value to my users, but I got to dictate that their work that uses it is also GPL (or they can pay me for a different license). I want that choice when it comes to my work being used as ML training data.
I think it will be great if they can create some mechanism to compensate people for their data, I just suspect many people conflate the value of their data as training data and say how much they might charge a client to write some similar code.
This only true if you select the most likely move instead of sampling from the probability distribution over possible moves. In the latter case there is no reason for there to be a wisdom of the crowd effect.
If you sample from the probability distribution you are modeling, there is no reason it shouldn't play like a 1100 player.
Well one reason is that it still won't be a 'single' player - what they've done here is like having a group of thousands of 1100 players vote on a move. What you're suggesting is to then pick a random player each move and go with them. There's no consistency, maybe on move 10 the player is blind to an attacking idea, but then on move 11 suddenly finds it...
I think they are saying, if your neural network was probabilistic and you thought there was a 90% chance of someone doing move A, but a 10% chance of move B, then you shouldn’t always get move A if it was human like - you would sometimes get move B.
I.e. most of the time if you leave your queen hanging and under threat your opponent will take it, but sometimes they just don’t see it. That’s the difference between playing a bot and a human a lot of the time - humans can get away with a serious blunder more often at low level play.
That's exactly what I'm saying - except more like the model is saying there's a 90% chance that a randomly chosen player at this level would make the move.
Yes, if you don't condition on the past moves then the distribution you're modeling is where you randomly pick a 1100 player to choose each move as you say. What I'm saying is that there will be no wisdom of the crowd effect.
> In the episode, the crew of the Enterprise visits a planet engaged in a completely computer-simulated war with a neighboring planet but the casualties, including the Enterprise's crew, are supposed to be real.
They did that in a Star Trek episode, more or less. "You have been assigned as dead. Please report...etc" so they could preserve infrastructure, which is so much more valuable than human life.
I don't really understand your point, you don't have to use it to generate the next sentence or paragraph of your story. You can it directly to generate ideas for what comes next by asking it to complete a summarization of the plot.
I understand the thought, and if you are unfamiliar with the process of writing a novel it makes some intuitive sense.
If that method were effective, we would not have to wait for a machine learning algo to make use of it. There has rarely been a time in our literate world where encyclopedic catalogues of plots and plot devices haven't been available. If one could throw a dart at Polti's 36 dramatic situations to understand their story better, or write their way out of a jam then I'd be more inclined to believe that you could use GPT-3 to muddy your way through a draft. This is not the case, however.
// Edit, addendum:
If I had GPT-3 generate a synopsis for me, based on a corpus of my work (let's say) I would have before me a framework that loosely adhered to my conventions and internal logic, however it would still be deep in that uncanny valley as any longer story from GPT-3 ends up being. The bulk of the work would be in reconceptualizing the generated synopsis into something that contained real, cohesive themes and character development. The project itself would likely be as much work as writing from scratch, but would also be an art project of sorts.
Novels are far more complex than most people assume. If you compare to movies or television, you have to take direction, cinematography, and production into account, rather than just the screenplay.
Maybe that sort of thing can be delegated to a GPT like algorithm, maybe GPT-4 will obsolete the novelist and the auteur, but I kinda doubt it.
Imo these are researchers. Their job was to validate their algorithm for doing generative super-resolution on a dataset, they chose the largest and most well-known dataset, it worked reasonably well on their dataset. The model itself is not productive ready for at least the reason that the dataset is not representative. This is ubiquitous in ML papers, they validate their idea on not completely realistic but widely available datasets. The outcome is a piece of knowledge about the behavior on that dataset not a product.
The point that many are making is that this is a myopic view of research. Why is the generic goal to optimize against some particular dataset? That ends up being a narrow and unhelpful goal.
As someone put on Twitter : we should be rethinking the meta learning algorithm of the ML academic sphere, and a leader like Yann is the kind of person who should be spearheading that.
Proactively doing the things that the ethicists are calling for. Use Model cards for any model. Actively think about and discuss the limitations the model, intended uses, etc. That goes for everyone.
For someone like Yann specifically? Publicly state that the ML scene is optimizing in a myopic way, and invest in doing so less myopically. For example, I think the translation space has a clear goal and the right goal and is making strides in improving language models in many useful ways.
Ultimately, if there aren't ethical and useful ways to apply facial recognition, leaders should be steering people away from those research topics.
Maybe it would be helpful if you gave an example of the simplest python function it won't be able to synthesize, and if/when they release the code GPT into the API we can test your prediction.
