Hacker Newsnew | past | comments | ask | show | jobs | submit | jaybaxter's commentslogin

X | Machine Learning Engineer - Community Notes | SF, SJ, Seattle, LA, NYC | All Levels

The role: build scalable production machine learning systems and infrastructure, and improve our open source algorithm: https://github.com/twitter/communitynotes

Community Notes has been discussed here on HN a number of times e.g. here: https://news.ycombinator.com/item?id=37292041

Apply here: https://twitter.wd5.myworkdayjobs.com/X/job/San-Francisco-CA...

There are many other positions at the company too, see: https://careers.x.com/en


I wouldn't say they were being childishly subversive -- it is very unclear that paper bags are better than plastic bags for the environment.

"Manufacturing a paper bag takes about four times as much energy as it takes to produce a plastic bag"

"Studies have shown that, for a paper bag to neutralize its environmental impact compared to plastic, it would have to be used anywhere from three to 43 times."

https://education.nationalgeographic.org/resource/sustainabl...


If you're going through the effort of carrying your own bags, why use a disposable bag at all?

Bringing up the harms of a paper bag is not really pertinent.


But they rot and don't end up in the sea.


It is very confusing to me how this got painted as pressure solely from the right. Ackman seemed like the driving force behind it, and he has mostly donated to democrats: "Since 1998, with few exceptions to select Republicans, Ackman has mainly donated to Democratic election candidates. Ackman has donated more than $600,000 to Democratic campaigns"

https://www.influencewatch.org/person/bill-ackman/#:~:text=2....


There is a type of advertising called "brand advertising" where the goal is more about brand awareness (e.g. think of a Coca-Cola superbowl ad) than "direct response advertising". Such advertisers don't really care if you click on ads, but they still want to reach high-value users.


I’m pretty sure most meta-originated ads are direct response. At least most of the ones I’ve seen. Most brand advertising shows up in places where engagement is hard (eg TV, billboards) while direct response is perfect for absentmindedly scrolling (you don’t feel bad clicking away).


The code is open source: https://github.com/twitter/communitynotes

The notes and rating data is released to the public every day: https://twitter.com/i/communitynotes/download-data

Feel free to run the matrix factorization code on the data, and then try to interpret the resulting latent dimension it finds for yourself! And also, you can read the code to verify that it really is running a matrix factorization rather than hardcoding a particular left/right split.


Indeed, it does this!

>For a note to achieve a high intercept term (which is the note’s helpfulness score), it must be rated helpful by raters with a diversity of viewpoints (factor embeddings)

https://communitynotes.twitter.com/guide/en/under-the-hood/r...


How come? Just long hours or something else?


Long hours and toxic, high-risk environment.



It started rolling out to everyone in the US in early October, and first version launched at smaller scale back in January 2021

https://twitter.com/birdwatch/status/1578004575990202370?s=4...


Timing feels really poor. In the middle of an election while Twitter was about to undergo a change in ownership.

I wonder if it would have seen the light of day if they waited until after the acquisition though.


Hi! Birdwatch ML Engineer here-- these context notes are from Birdwatch. They are written and rated by users, and the notes are only added as context to Tweets if they are rated highly enough by multiple raters who have tended to disagree in the past. The core algorithm is open source as well as all of the data, and there is lots of public documentation about it too:

https://twitter.github.io/birdwatch/

https://twitter.com/birdwatch/status/1585794012052611076?s=2...

https://arxiv.org/pdf/2210.15723.pdf


Really admire your work man.

I honestly felt nothing but contempt for the oligarchic structure of fact checking. Facebook hired organisations like the Washington Post to do their fact checking. I have not forgotten how many people there lies killed in Iraq. I don't forget how the journalists at the time were willing to compromise their ethics in exchange for White House access and exclusive stories from the front lines, simply because journalistic interest is not the public interest. We also saw more recently during the pandemic incompetent journalists without adequate medical training censoring BMJ articles from Facebook: https://www.bmj.com/content/376/bmj.o95.

You are making one of the only tools I can see that can possibly make "fact-checking" a reality in a nominally unbias way that can be broadly accepted by the public. A democratic fact-checking system for a largely democratic website. Even if birdwatch will get things wrong, I appreciate the effort to reform this institution of "fact-checking" with something righteous and moral. No need to touch on my spicy political views, but I think this is a transparent, scalable, fair tool that can truly be embraced by the entire society rather than simply leading to a partisan divide.

Your system understands I'm on social media because I want to hear what the average person has to say, and I would go to a news website if I wanted to get a news websites take on what the facts were. To me, if a news website wants people on twitter or Facebook to pay attention to their "fact check" editorial, they should do it the same way everybody else has to. By creating compelling writing that gets attention and that people agree with.


I'd be interested to hear your opinion:

Do you think there is a risk to "fact checking" gives the public a false sense of knowledge, and reduces certain important skills?

It seems apparent to me that, in the process of a manual substation of truth, through whatever means of research an individual chooses to check the validity of a subject, they accidentally learn more during the task of that research verification both directly through the claim that was made, and as well in that field in a more general sense.

For example: if two users are arguing over a Bible verse, and one side wishes to verify the claim the other side made, the traditional method would mean anything from reading a Wikipedia entry and a few related sources, to a book on the subject, to some other means. During the search for the specific thing they were after, they would also learn about: the general historical context of that period, other related verses, events of a similar nature throughout history, cultural peculiarities, linguistical differences, etc.

On the other hand, if "fact checking" is simply a thing done on a persons behalf, not only is the accuracy of the check a concern, but the act will not encourage learning about that related context. The recipient will not have a reason to learn of those other things, and might never learn or maintain that valuable skill of research and verification.


I feel "fact checking" as first pioneered by politifact (if you don't count snopes) was a significant advance in journalism and I recall they did hold a substantial number of left wing journalists and talking heads to account, bias or not. The biggest change was that journalism traditionally expected you to trust the veracity of a story based on a journalists reputation. After journalists reputations largely got smashed by the Iraq War, I really appreciated Politifact's commitment to at least stating what their sources of information actually were. It felt like a more transparent form of journalism you could "Fact check" yourself. The articles WERE still fundamentally subjective opinion pieces, but they were at least based on sources you could actually look up yourself, and often with these fact check articles I treat them like wikipedia use them as an index of sources.

I'm not against fact-checking persay, I'm more against the artificial promotion of certain parties influence through forced editorialisation. I feel birdwatch holds the promise of a better way to accomplish the same end.


I think there is nothing wrong with fact checking the problem is that the fact checking that is being done often has a bias. One example that comes to mind was the Hunter Biden laptop that was dismissed as Russian propaganda by the fact checks. Turned out it was completely true.


Some items found on it were certainly his. How many lawyers would care to go into court with a piece of evidence having a chain of custody like that laptop's?


I’m not a lawyer but he is on video in 4K doing illegal activities. I think it would be an open and shut case.


>turns out it was completely true

Depends on what you mean by “completely true.” For instance, I heard all kinds of wild/absurd accusations like he had child porn all over it. If that were “completely true,” he’d be in cuffs. You need to give more context here.


Sure, let me be clearer. The media said the laptop and the videos were Russian misinformation. Turns out that was his laptop and that was him in those videos.


I'm going to be honest, this doesn't clear up much at all. What videos are you talking about? And when did Hunter et al say his laptop literally didn't exist?

What was disputed was what the repair tech said he found on it, as well as the fake news/gossip Hannity and co kept repeating without any evidence. Remember when a courier "lost" the supposedly incredibly damning and totally-proves-something-about-Hunter evidence en route to Hannity that they conveniently had no copies or backups of any kind? We still don’t even know what they allegedly contained!

Russian accounts absolutely amplified and generated misinformation around Hunter Biden in an attempt to hurt Joe Biden's election chances. They have consistently engaged in this behavior during our elections. There is a consensus among our intelligence organizations, it's old news.

Please present concrete claims and proof if you're going to say something is "completely true."

https://www.theguardian.com/media/2020/oct/29/fox-news-tucke...


>what videos

The videos of him smoking crack and negotiating with prostitutes in Ukraine.

>when did Hunter Biden deny the laptop

Joe Biden went on stage to debate Donald Trump where he publicly said that the laptop was “Russian disinformation”. Hunter Biden didn’t really deny it though he said he “wasn’t sure if it was his”.

Twitter refused to post the NY post story as well as banned users mentioning the Hunter Biden laptop scandal.

Obviously, this was a cover up. The laptop was his.


Sounds like you should have no issue sharing these alleged videos or at least proof of their existence. Or any of your sources. I’m also not sure why Hunter’s well-documented and acknowledged addiction is such a fixation for the right.

I’m done man. Have a good one. I’ll gladly read any sources you send along.


This is not a left or right issue. It’s the media and big tech covering up the facts that’s an issue. Sounds to me like you have a bias.

>videos don’t exist

Wait, do you seriously think these videos are fake? They were all over Reddit.

I can send you the link. I can’t post it here since it has nudity but I’m more then happy to send it to you. Is there an email or way you would like to receive these vids? (MP4 files).


Although I think that most fact checking is biased, I am less concerned of bias, as long as people have a means and an ability to determine the truth of the thing. My concern then isn't with where bias exists, or in what direction, but instead with what the effect of fact checking results in. I think it would probably be a healthier outcome for truth learning if a "fact checker", whether it be a person or a machine, would instead point the user to instructions of how to research "For the field of (whatever) you will want to consult (this) area of research.", rather than "According to XYZ, (some person) said it's (that)."

Like, if I wanted to learn about something regarding the financial affairs of Europe in the 1300s, someone could just link me a quote that Professor so-and-so said something. But what if that professor is generally seen as a quack in that field? What if, after making that statement, he corrected himself a week later? What if that quote is simply decades old, and new information has come to light since then? What if my interest was specifically in one area of Europe, but his statement encompassed multiple countries, and parts of the Middle East? What if the statement being made had an assumption of "excluding other factors regarding wars A, B, C and plagues, 1, 2, 3" but was not in the quote that I was given?

These are all things that one might accidentally pick up and learn about during exploration of the truth, but be completely unaware of needing to know that when they first started.

To give a real world example of what I mean, consider this: A prominent New Testament scholar is Dr. Bart D Ehrman. In most debates I have seen posted on YouTube regarding whether The Bible is a trustworthy source will reference Dr. Ehrman at some time or another, by those in favor of Christianity and scriptural accuracy. The thing is, Dr. Ehrman is an strong opponent of Christianity and outspoken Atheist. Dr. Ehrman is a very big name in the field, and has a lot of criticisms against The Bible itself, but he also strongly affirms historical proof of Jesus' existence, life, and death - he's written at least one book on the matter[1]. If someone were to simply see the quote he has made, "but he did exist, whether we like it or not." the immediate assumption about his stance and belief is that he's a Christian, and therefore biased.

It's the loss of this context, this underlying breadth that I'm concerned with. That fact checking, in a very real sense, removes both the onus and need to read and determine for ones own self. That fact checking does too much, and gives too little; the worst of both. A fact without the background, a cause without explanation.

[1] https://www.bartehrman.com/did-jesus-exist/


Ironically their site tagline is "democracy dies in darkness"


Was there a specific instance you can point out where the Post got its facts wrong, and didn't correct it?


[flagged]


Was there a specific instance you can point out where the Post got its facts wrong, and didn't correct it?


An org thats concerned about democracy should not let ads which lie and opinion pieces without fact checking not allowed.

That's what I expect but You may have diff standards


Was there a specific instance you can point out where the Post got its facts wrong, and didn't correct it?


Yes, they printed the lies about WMDs and put out loads and loads of articles in support of the war.


At that point in time as Sadam had just used WMDs against the Kurds and many observes thought it likely he had not all of a sudden stopped having them. That no one found any WMDs later on does not make reporting what intelligence agencies said at the time “lying”. Do you have any information that the Washington Post was reporting something they knew to be untrue _at the time_?


Taking what intelligence agency’s say at face value isn’t reporting, you’re just a state mouthpiece.


At the time it was very reasonable to assume Saddam had (more) WMDs. Stating otherwise is suffering from hindsight bias, there is really no reason to assume malice here.


That's just not true. The "source" for US intelligence was called Curveball and the information came from Germany. The German intelligence service, BND, warned the US multiple times that the information was made up. Even the CIA's head of the European division Tyler Drumheller said that he warned George Tenet that the information was unreliable[2]. Tyler Drumheller also said that "everyone in the chain of command knew exactly what was happening"[3].

The informant was called:

- "crazy … out of control" (by German Intelligence)

- "congenital liar" (by Friends)

- "an alcoholic" (by US physician) [4]

UN weapons inspectors said at the time that the US is deliberately ignoring their information.

They KNEW for a fact that the information was false. It's not like they then found out that the information was bogus and that hindsight is 20/20 or something. It was known at the time.

It was also public knowledge in countries like Germany. Do you really think The Washington Post and other outlets don't check what the Germans had to say about all of this considering the information came from Germany? Not suspicious that the country where the information supposedly comes from doesn't want to support the war at all? Nope. The Washington Post and others didn't care. They wanted that war and supported it.

[1]: https://www.welt.de/politik/ausland/article13569264/USA-habe...

[2]: https://www.theguardian.com/world/2011/feb/15/curveball-cia-...

[3]: http://articles.latimes.com/2005/apr/02/nation/na-intel2

[4]: https://www.theguardian.com/Iraq/Story/0,2763,1451167,00.htm...


That's one source of information about WDMs being false, and that may indeed have been knowable at that time. But lack of proof does not prove the opposite, it does not prove that Saddam did not have WDMs.

Saddam had used WDMs on multiple occasions in the past and it was not unreasonable to think he would use them again, even without any proof he had any more. It's not all that hard to hide those weapons if you are a dictator. The US made the mistake of presenting this incorrect piece of information. But reporting reasonable sounding information from an intelligence agency, even if that later turns out to be false, does not make the Washington Post a "liar" or a "state mouthpiece", as the grand parent claimed.


> But reporting reasonable sounding information from an intelligence agency, even if that later turns out to be false, does not make the Washington Post a "liar" or a "state mouthpiece", as the grand parent claimed.

Sure sounds like being a state mouthpiece to me. That's kind of what uncritically echoing what a murderous government tells you, while refusing to give the time of day to anti-war voices like a good little lapdog is.

In regards to them being liars - kust read their reporting from the time, such as this article titled "Irrefutable"

https://www.washingtonpost.com/archive/opinions/2003/02/06/i...

"AFTER SECRETARY OF STATE Colin L. Powell's presentation to the United Nations Security Council yesterday, it is hard to imagine how anyone could doubt that Iraq possesses weapons of mass destruction. Mr. Powell left no room to argue seriously that Iraq has accepted the Security Council's offer of a "final opportunity" to disarm."

This reporting they did was a lie that helped sell a war under false pretences.


It wasn't UNREASONABLE but I wouldn't say it was VERY reasonable. At the time a number of traditional US allies did refuse to (openly) participate in the "Coalition of the willing". I think if you got your news from non-American and non-British sources at the time, I think there was far more uncertainty and doubt that Saddam had such weapons. I have heard this line before "Everybody at the time thought it" but I genuinely saw most people around me calling bullshit. There was a big sense of a grasping of straws from US intelligence, such as aluminium tubes being great evidence of nuclear development, and "we don't want the smoking gun to be a mushroom cloud".

In fact it was remarkable just how much more bias the American media was then the global media at the time, and it doesn't seem to me like the American media had any special reason to be more confident that Saddam was nuclear than any of the media in the rest of the world. My theory wasn't that American mainstream media outlets like WaPo and NYT were going out of their way to start a war under false pretences, it was more that they had a lot of incentives to be bias in a pro-war direction.


I live in Europe so I did get my news from non-US media sources. There were a lot of reasons not to go to war, especially in the way the US did. There was no UN backing yet and there was no good plan what to do with the country once Saddam was gone. But claiming you knew that Saddam had all of a sudden given up all his WDMs doesn't seem like a very strong one. This man violated countless UN resolutions about WDMs in the past, and now he all of sudden went straight? How is that a reasonable assumption? Also, it's not impossible he did have them after all and just hid them well.


The Guardian's comment section was called "Comment is Free," but was heavily moderated.


Gratis vs Libre, perhaps. It’s a shame English is so stunted in this regard.


Now do vaccines


This is one of the coolest things I've seen out of twitter.

Now if only if the feed would stop disappearing tweets in the middle of reading them, that would be great.


Disappearing tweets without site reloads?

I guess this can't happen with the Nitter front-end. Just use LibRedirect.

https://libredirect.github.io/


Yes. As in I'm looking at the screen reading a tweet, and it vanishes in some animated fashion with me not doing anything. It's not gone, but I have to know how to find it to see it again.


Isn't that one of peculiarities of algorithmic timeline? At least chronological one is the same after reloading.


A lot of apps (I don't really use Twitter, but assume this of it) do this thing where you're at a page/view you'd like to return to later, so when you actually do return later (either by switching between apps or unlocking your device) then all the content you wanted to continue is right there as expected, ready to go, then the app unexpectedly triggers an unwanted refresh animation then takes you to the homepage or if not that, it could be impossible or unclear how to get back to what you were just doing/reading. Firefox for android doesn't do this, nextdoor does.


What I'm seeing is individual tweets disappearing while looking at my feed. And it's animated + 144hz monitor so I can see them fly away into the either. Sometimes while reading them and about to click on them. And then I can't. It's really annoying.


Yeah, it's a common complaint, seen it on HN and Twitter quite a few times. In fact, there was recently a joke tweet with 95k likes about it:

>Twitter laid me off today. If you know of any open positions for project managers and senior software engineers, let me know. I was the PM in charge of that feature that shows you a good tweet before the whole timeline refreshes so you can't find it

https://twitter.com/bantablack/status/1586080710468648960


>They are written and rated by users, and the notes are only added as context to Tweets if they are rated highly enough by multiple raters who have tended to disagree in the past.

Interesting, I remember someone on /r/slatestarcodex having the idea to rate/sort reddit comments by that metric, which I dub "sort by should-be-uncontroversial" (because normally-disagreeing people all think it's correct).

(Then again, it almost might incentivize politician-style vacuous/feel-good statements.)


>it almost might incentivize politician-style vacuous/feel-good statements

They for sure would since there's a word for that: demagoguery.


Then those of us who hate "politician-style vacuous/feel-good statements" would down-rate them.


Yeah, but that does nothing on Reddit. Those "politician-style vacuous/feel-good statements" are already highly upvoted there.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: