I'm the developer of an Android calculator, called Algeo [1] and I wonder which part of it that makes it feel like slow/not snappy? I'm trying to constantly improve it, though UX is a hard problem.
This seems to be an expression mode calculator. It simply calculates the result of an expression, which makes it like the other 999 calculators in the Play Store.
Classic algebraic calculators are able to to things like:
57 x = (displays the result of 57x57)
3 = (repeats the multiplication and displays 57x3)
[+/-] MS (inverts the result and stores it in the memory without resetting the previous operation)
7 = (repeats the multiplication for 7 and displays 57x7)
7 [1/x] = (repeats the multiplication for 1/7, displays 57/7)
It doesn't have to be basic arithmetic, this way you can do complex numbers, trigonometry, stats, financial calculations, integrals, ODEs etc. Just have a way to juggle operands and inverse operators, and some quick registers/variables one keypress away (see the classic MS/MR mechanism or the stack in RPN). RPN calculators can often be more efficient, although at the cost of some entry barrier.
That's what you do with the classic calculators. Often, you are not even directly calculating things, you're augmenting your intuition and offloading a part of the problem to quickly explore its space in a few keypresses (what if?..), give a guesstimate, and do some sanity checks on whether you're in the right ballpark, all at the same time. Graphing, dimensional analysis in physics, error propagation help a lot in detecting bullshit in your estimates as quickly as possible. If you're also familiar with numerical methods, you can do miracles at the speed of thought. Slide rules were a lot like that as well.
People who do this might not be your target audience, though.
Pytorch and tensorflow is pretty big. It implies all state of the art research code is not written in Julia, so it is a non-starter for any neural network based project.
That is just total nonsense. You can of course do neural network research without pytorch and tensorflow. Julia is especially good, with Flux being the most flexible Neural Network library which exists.
Pytorch and tensorflow are important and a very good reason to use python or C++, but those two being unavailable is more impactful for the industry, of course your research might need them, but it might very well not.
Because it only increases high single digit each release. If they keep up the 10% improvement for the next 10 release, we will reach a speedup of around 2.5 times. That's very small, considering how Python is like 10-20 times slower than JS (not even talking about C or Java like speeds).
‘O’ has an interesting backstory. The shape comes from phonecian ayin and has the meaning of “eye”, from which it takes its form. It probably had a sound close to the Arabic ayin (a kind of guttural growl where you half choke yourself).
However the Greeks were probably just as confused by that sound as non-Arabs are by it today, so they instead gave it a vowel sound.
‘A’ had a similar story. In Phonecian it represents a glottal stop and has the name ‘alep. However a glottal stop at the start of a word is quite hard to hear if you aren’t used to it, so the Greeks seemingly assumed that the letter represented the second phoneme of ‘alep, an /a/ sound. Hence the alphabet was invented, a mixture of vowels and consonants with equal status.
It's really a lucky happenstance that Semitic languages have enough gutterals that could be repurposed as vowels; if these had been lacking, or if Greek had needed them as consonants, then it's easy to imagine a sort of incomplete consonant-only orthography coming into use, perhaps even persisting to Latin alphabets in the modern day. Greece's first introduction to writing (linear b) didn't stick and doesn't seem to have been used for anything other than palace records, arguably because of deficiencies in that script, might literacy in the West have similarly been stunted by scripts that could not show vowels?
Losing the vowels works in languages like Arabic because of the structure of the language, with consonantal clusters (e.g. ktb, tlb) being semantic “roots”. So you usually know from context, and if you don’t know the absolutely precise word, you know the meaning, which can sometimes be more informative.
And Greek too has two versions of the lower-case sigma: σ (whose usage corresponds to that of the long s) and ς for use only in the word-final position (corresponding to the terminal s as distinguished from the long s).
This rings so true with me. I have a side project [1] that I try to promote and it always feels hard. Where do I start? Should I post comments in random subreddits? Should I write blog posts that may nobody read in the end? Start doing ads? I think having the multiple options kinda freezes me and it is so hard to start doing something.
Congrats! It's about the same I make with Android with my app (a graphing calculator) [1]. Everyone says on Android you can earn less, I'm curious what's your experience will be.
I don't really get the first part of your comment. You say an NN is "just" compression + kNN and does no representational learning. But finding a compression (a transformation in other words) that makes kNN feasible on the data is exactly what people mean when they say it finds a hidden representation. It is a highly non-trivial task: e.g. simple distance in pixel space between images would get you useless results.
People have the notion that a latent representation in the animal sense, ie., a concept -- is the same thing as your "representation" in the NN sense.
That's not the case. You're right that if I find a predictive compressions of faces, say F1...n then they arent literal "rememberings". And they seem to be able to participate in a decision process (eg., classification) which doesnt seem to target pixel patterns.
However I think this is kind of an illusion. What `F1..n` are, are ambiguous pixel-space projections of the abstraction which isnt present in this projection. When I have the concept "this type of face" I can reason with it beyond similarity in pixel-space.
When we form representations we arent restricted to reasoning with them in only one space (eg., how faces look as pixels). We (perhaps superstitiously) impart to machine "representations" an actual depth which they lack.
They are templates derived from the spaces they live in, eg., pixel-space; and have only the properties that space affords (eg., pixel-geometry). Reasoning beyond that space, and those properties, doesnt work. People think it does. This is the illusion.
Templates derived from this data, that we provide, function like actual representations because we simplify the world for the machine -- and prepare its environment so that its pixel-space templates are good enough.
> When I have the concept "this type of face" I can reason with it beyond similarity in pixel-space.
I think there are at least two possible things that might going on here:
1. we're "trained" on non-pixel data (to use the same framing) and so it seems obvious that we would reason with a concept like "this type of face" in a non-pixel space.
2. the experience of "reasoning with it" is an illusion, and is merely the subjective experience that we have when our brains do stuff with whatever their underlying representation is. That is, we may have no real knowledge of what space our own "face model" is built on, what it represents, what properties it considers.
So, another option, is that our concepts arent really about their targets.
For example, my "pen" is actually a bundle of largely motor techniquess (also: emotional regulation technquies etc.) which is about how I coordinate myself.
In a sense all my concepts are about me. We might say that in congition we abstract these "primal concepts" into properties which are then about their targets, sure. This is why cognition is a bit of a charlatan.
In being able to regulate my own imagination, motor skills, emotions etc. with these techniques, I can actually discover new things about the "abstract concepts" that they imply.
Since my "pen" isnt really anything like "a pen" nor even my experiences of pens... it is rather, "a way of me moving everything within me" -- I can simulate these movements and discover new things about their implied abstractions. Indeed, I can do this in the world: I can explore actual pens by moving my body differently.
This is one of the big issues with the AI paradigm as it has always existed: researchers are still talking about environments as if they had properties which were already there. Everything is formualted as-if the problem were solved. The world is just some bundle of facts (data, propositions, etc.) and representations are just subsets of these.
This misses, you know, the world. The thing that hits you. And it misses, entirely, what happens when it hits you.
Sounds like a re-visit of a lot of the ideas in "situated action", popular in some AI circles in the 90s. That also included concepts about how we reduce cognitive load by storing information (and procedure) in our built environments.
I'm not, though, that I agree about the pen example. I mean, you're completely correct in your description of all these different elements of your "pen". But to the extent that a concept of/about something is really never anything like the thing, it's not a particularly important aspect of concepts. I think that what's important is that all of these embodied, semi-reflective elements of your "pen" concept sit in parallel with some abstract concepts about pens. Pens do have properties that are already there, but in many contexts as you note, these are less important than the ones embodied by you.
I suspect we'll find that these abstractions are no where within us. We dont have ML-like templates of stuff.
Whenever we need to reason about the properties of pens, our coordinative bodily structure generates these abstractions for cognition to operate on. They're ephemeral, and live only when cognising an issue.
To model cognition then, is to model only the symptom of intelligence; not the actual process itself.
Slightly related: I got the first C compiler restored and ported to modern C [1]. It runs on Linux and can call libc functions, though it is restricted to 32bit mode.
Actually, MobileNetV3 is a supporting example of the bitter lesson and not the other way round. The point of Sutton's essay is that it isn't worth adding inductive biases (specific loss functions, handcrafted features, special architectures) to our algorithm. Having lots of data, just put that into a generic architecture and it eventually outperforms manually tuned ones.
MobileNetV3 uses architecture search, which is a prime example of the above: even the architecture hyperparameters are derived from data. The handcrafted optimizations just concern speed and do not include any inductive biases.
"The handcrafted optimizations just concern speed"
That is the goal here. Efficient execution on mobile hardware. Mobilenet v1 and v2 did similar parameter sweeps, but perform much worse. The main novel thing about v3 is precisely the handcrafted changes. I'd treat that as an indication that those handcrafted changes in v3 far exceed what could be achieved with lots of compute in v1 and v2.
Also, I don't think any amount of compute can come up with new efficient non-linearity formulas like hswish in v3.
Back of a napkin calculation: showing OP 100 ads of blinds probably costs 0.1$ in the US. The cost of a blind is 100$ (I have no idea, don't live in the US). So if the probability of buying a second blind is at least 0.1% it was worth running the campaign.
[1] - https://play.google.com/store/apps/details?id=com.algeo.alge...